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ABSTRACT 

A third-party national evaluation of the basic grants 
section of the Adult Education Act was conducted in September 1990. 
The study drew on data from the following sources: 1990 mail survey 
of all federally supported local service providers; longitudinal 
study of local programs that provided information on the 
characteristics of more than 22,000 adults enrolled in local programs 
from April 199i to April 1992; and telephone survey of a subsample of 
nearly 6,000 clients 6 months after they left the program. The 
samples were weighted to make them nationally representative. The 
program was serving between 2.6 and 3.2 million clients annually 
(approximately 467. in Engl ish-as-a-Second-Language programs, 307. in 
adult secondary education, and 247. in adult basic education) . The 
program was credited with improving clients* reading, employment, and 
self-concept levels. Its population was increasing by about 
two-thirds more than its capacity. Appendixes constituting 
approximately 507. of this report include the following: table 
detailing the study objectives and reports where they are addressed; 
telephone survey instrument; information on sample design, data 
collection, and standard errors; persistence tables; methodology of 
the learning gains study; and 35 selected updated tables from second 
interim report. Contains 54 tables/figures and 33 references. (MN) 
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EXECUTIVE SUMMARY 



Background 

In September 1990 the U.S. Department of Education contracted with Development 
Associates, Inc., to conduct a national evaluation of the basic state grants section of 
the Adult Education Act. This report summarizes findings and conclusions with 
respect to client enrollment and attendance, client educational and employment 
outcomes, and the impact of the program on the country's continuing literacy needs. 
Three previous reports from the study contain additional detail. 

Methodology 

The study draws on data primarily from: (1) a mail survey of all federally supported 
local service providers, conducted in the faU of 1990; (2) a longitudinal study of local 
programs, which provided information on the characteristics of over 22,000 adults 
who enrolled in local programs during a 12-month period (April 1991 to April 1992) 
and on the amount and type of instruction and other services these clients received 
for 18 months after they erjrolled; and (3) a telephone survey of a subsample of 
almost 6,000 clients 6 months after they left the program. The samples have been 
weighted to make them nationally representative. 

Major Study Findings 

Patterns of enrollment and persistence 

• the program serves between 2.6 and 3.2 million clients armually, with 
about 2 million of them newly enrolled; 

• about 46 percent of clients enroll in ESL, 30 percent in ASE, and 24 
percent in ABE; 

• about 15 percent of clients who enroll in the program never actually 
receive instruction; 

• the median number of hours of instruction for clients who receive 1 
hour of instruction or more is 58 hours; 

• the median number of hours of instruction clients in ESL receive is 
about 3 times that of those enrolled in ABE (113 and 35 hours, 
respectively) and 4 times that of clients enrolled in ASE (28 hours); 
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• about 76 percent of the total number of instructional hours received is 
by clients in ESL, about 13 percent in ASE, and 11 percent in ABE; and 

• the strongest predictors of persistence are: 

— > the presence of support services that clients actually use; 

the receipt of instruction during the day as opposed to evening 
hours; and 

— > the type of learning environment in which clients participate (but 
the type of environment that was most predictive was not the 
same for all three instructional components). 

Client-related outcomes 

• Clients who took pre- and posttests made significant improvements in 
their reading achievement: 

-> ESL clients gained an average of 5 scale score points on the 
CASAS reading test; and 

-> ABE and ASE clients gained an average of 15 and 7 scale score 

points, respectively, on the TABE reading comprehension subtest. 

• On the basis of interviewing clients 6 months after they left the program 
we found that: 

— > a majority believed they had been helped to improve "a lot" in at 
least one area of basic skills; 

there was a net increase of 6 percent in employment following 
participation in adult education, with ESL clients reporting 
having benefitted the most; 

--^ between 11 and 30 percent of the program's ASE clients (or, 4 to 
12 percent of all clients served) were helped by adult education to 
complete their secondary education; 

approximately 70 percent of the program's participants who had 
goals related to improving their self-concept were helped in that 
regard; 
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-> there was a net increase of 15 percent in the number of former 
clients who read more frequently to yoimg children in their 
household; and 

-> approximately 70 percent of the clients benefitted from 

participation in adult education with respect to at least one of the 
three major purposes of the Adult Education Act, with the 
majority of clients (58 percent) indicating these benefits to be in 
the area of improving their basic education skills. 

Reasons clients leave the program 

• 41 percent of the participants report they left the program because they 
had completed their course of study; 

• a plurality of clients (45 percent) left for reasons external to the 
program, such as employment, health or child care problems; and 

• approximately 7 percent of the clients terminated instruction because of 
dissatisfaction with their adult education program. 

Program results 

By relating estimates of the size of the program's target population to 
information on client enrollment and attendance the report shows: 

• the program serves substantially more ESL clients per 1,000 members of 
the target population than members of the other two subgroups (i.e., 
ABE or ASE); 

• depending on the time of year, between 16 and 25 percent of programs 
report that they have waiting lists of clients they are tmable to serve, 
with most of the clients on these lists waiting to begin instruction in 
ESL; and 

• the program's target population increases by about two-thirds more 
each year than the program is able to successfully serve, with the 
greatest area of xmmet needs being adult secondary education. 
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Chapter 1 
INTRODUCTION 



The Adult Education Act was passed by Congress in 1966, and since that time 
the federal government has provided funds to states to help support a system of local 
adult education service providers. In 1991 the National Literacy Act broadened and 
strengthened the adult education program by incorporating as its Title m the basic 
state grants provisions of the earlier act. These grants are intended 

to assist States to improve educational opportunities for adults who lack 
the level of literacy skills requisite to effective citizenship and 
productive employment, to expand and improve the current system for 
delivering adult education services including delivery of such services to 
educationally disadvantaged adults, and to encourage the establishment 
of adult education programs that will — 

1. enable these adults to acquire the basic educational skills 
necessary for literate functioning; 

2. provide these adults with sufficient basic education to enable 
them to benefit from job training and retraining programs and 
obtain and retain productive employment so that they might 
more fully enjoy the benefits and responsibilities of citizenship; 
and 

3. enable adults who so desire to continue their education to at least 
the level of completion of secondary school. 

In September 1990 the U.S. Department of Education embarked on a major 
national evaluation of the basic state grants section of the Adult Education Act. The 
central purpose of this study was to evaluate the potential of programs supported by 
that section "for significantly reducing deficits in the adult population with respect to 
literacy, English proficiency, and secondary education." (See Appendix A for the 12 
objectives of the national evaluation.) 

This report is the last of a series of four that present study findings. The first 
interim report (1992) describes the adult education service delivery system and is 
based on a mail survey, completed in December 1990, of all service providers and on 
more detailed information obtained from 131 programs during May-November 1991. 

The second interim report (1993) presents characteristics of clients who entered 
sampled adult education programs during the evaluation's 12-month intake period 
beginning in April 199L Program staff recorded client data on forms specially 
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provided for that pixrpose. The report also presents preliminary findings on client 
attendance and persistence. 

The third report (1994) describes patterns of attendance and factors associated 
with the persistence of adult education clients during tlie first 12 months following 
their enrollment in the program during the intake period beginning April 1991. The 
major data source for the report was the client update record, which provided data 
on attendance patterns of, and services used by, each client. 

This, the fourth report, focuses on client outcomes and on program impact on 
adult education demand and needs vis-a-vis the target population. This report uses 
the study's final data files relating to research described in the first three reports and 
includes telephone follow-up survey data collected from a subsample of clients 6 
months after they terminated instruction. After publication of the second and third 
reports, data on which they are based had to be adjusted mainly for two reasons: to 
incorporate late submissions and to revise sampling rates to adjust for actual 
participation in the study. These adjustments are fully reflected throughout this • 
volume, and their impact on tables in the second interim report is noted in Appendix 
F. The adjustments' impact on key tables in the third interim report is addressed in 
chapters 2 and 3. 

Published separately, an executive summary presents an overview of the 
findings and conclusions of all four reports and reflects the final data set. 

Overview of the National Evaluation 

The evaluation began with a Universe Survey, encompassing all local adult 
education service providers receiving Adult Education Act funds in the program year 
ending June 30, 1990. The results of that survey were used to obtain a nationally 
representative sample of 139 local programs for participation in the longitudinal 
phase of the study that began in April 1991. These programs agreed to complete a 
Comprehensive Program Profile Questionnaire and to provide data on the 
characteristics of a sample of clients who first enrolled in adult education between 
April 1991 and April 1992. They also agrec^d to supply data on the extent and type of 
instructional services those clients received for 18 months afier their enrollment. Of 
the local programs initially agreeing to participate in the evaluation, 131 actually 
began data collection and completed a Comprehensive Program Profile 
Questionnaire. The final data files consist of data from the 116 programs that 
completed the 12-month client intake period and provided client enrollment data in 
accordance with the study's sampling plans; 110 of these programs provided 
attendance and related update information on their sampled clients for the entire 30- 
month data collection period. 
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Before data collection began, the participating local programs were trained to 
complete forms and protocols. We also reimbursed programs for some of the costs 
associated with data collection. Throughout the data collection period, we provided 
technical assistance to participating local programs in data collection and monitored 
the quality of the data received. 

During the longitudinal phase of the evaluation, data were collected on the 
characteristics and experiences of entering clients, the amount and nature of 
instructional services those clients received for a period of 18 months, and 
characteristics of the service programs in which the new clients enrolled. Also, for 
subsets of those clients, data were collected — through telephone interviews 6 months 
after the clients ceased receiving instructional services — on basic skills and on clients' 
perceptions of program experiences and benefits. Exhibit 1.1 provides a brief 
description of tiie data collection instruments that serve as the primary sources of 
information for this report: the Universe Survey, the Comprehensive Program 
Profile, four types of forms on which program staff recorded data on individual 
clients, and the telephone follow-up survey. Appendix B contains a copy of the 
telephone follow-up survey; copies of the other data collection forms are included 
with the study's three interim reports. 

The study called for collection of a substantial amount of information about 
each client, and not all the data sought were received. To compensate for the 
incomplete information, we limited the data base to those programs and clients that 
supplied reasonably complete intake and update records and then took two steps: 

1, Adjusted sampling weights- We assigned clients sampling weights 
based on the likelihood of their being selected for the study. Each 
client's likelihood of selection was based on the probability that the 
client's program was selected, that the specific instructional delivery site 
was selected, and that any given client at the site was selected. Program. 
noru:esponse required us to adjust the originally assigned sampling 
weights. The effect of these adjustments was to maintain the nationally 
representative nature of the study's data base. The sampling weight 
adjustments, as well as estimates of standard errors for the data, are 
discussed in Appendix C. 

2. Imputed some responses-For a small number of variables, we have 
used other responses from the same client to impute missing data. 
Generally, however, we have elected to let sample sizes vary in different 
analyses rather than to maintain sample sizes by imputing data. 

Finally, we should point out that Client Intake Record: Part B and client 
attendance data could not be collected from two very large programs-Chicago and 
Miami. As described in Appendix C, we have investigated the potential effect of 



these missing data on our analyses and, where appropriate, have made additional 
technical adjustments to correct for biases in the data (C-7). 

Adding Perspective to the Evaluation: Some Challenges We Faced 

Implementation of tlie study involved numerous challenges. They serv^e to 
supply valuable additional perspective to the findings as well as provide useful 
lessons for future evaluations of adult education programs. 

What is a "program" for the purposes of the national evaluation? Despite its 
apparent simplicity, that question posed a challenge. For inclusion in the universe of 
adult education programs, each had to receive financial support through the basic 
grants provisions of the Adult Education Act. Receipt of such funds, however, did 
not adequately target the administrative entities that should be included in the 
evaluation. For example, more than one grant may be made to the same 
administrative agent, such as separate grants for the ABE, ASE, and ESL instructional 
components. Or sometimes a basic grant is awarded to a regional administrative 
service agency that has several subgrantees, some of which may be local school 
districts and other community-based organizations; and grantees exercise varying 
degrees of administrative control over the service delivery agencies. 

Also deceptively simple was the question: Who is a client for purposes of 
inclusion in the evaluation? We defined a client as a person registered to receive 
ABE, ASE, or ESL services directly supported by Adult Education Act fimds, or 
services similar in content to, and closely coordinated with, instruction directly 
supported by the Act. But how does one classify persons who receive ASE 
instruction in programs that received Act funds for use in ABE and ESL only? We 
classified them as clients for inclusion in the evaluation. And what is the status of 
those in programs that offered courses in vocational skills (e.g., electrical repair) and 
enrichment courses (e.g., parenting) only? Those persons were not included in the 
evaluation. Nor were those incarcerated in prison or jail. Generally, we also excluded 
clients in nursing homes. 

We did, however, include clients who lived in homeless shelters, halfway 
houses, group homes, and other intermediate detention facilities. Also included in 
the evaluation were clients who changed classes within the same instructional 
component (such as from beginning ABE to intermediate ABE), who changed 
instmctional components (for instance, from ESL to ABE), or who transferred from 
one instructional delivery site to another. The evaluation encompassed noncitizens as 
well, including illegal aliens. 

And at what point in their instruction were persons defined as clients by 
programs? Some programs consider as clients all who complete their intake or 
registration process; others do so only for those who receive at least 12 hours of 
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instruction; still others use a point in-between. The evaluation gathered information 
on those who filled out a program's intake /registration form, which is completed by 
the client's second day of instruction. 

Another challenge was the initial absence of information on program size, 
administrative organization, and instructional services offered. For example, a list of 
subgrantees was not available at the federal level, and information at the state level 
(where the operational definition of "local program" was not consistent across states) 
varied considerably in content and quality. Some programs did not have information 
on the composition of staff or the nature of the instruction provided at different sites. 
Nor did many programs have any idea of the number of adults newly enrolled each 
year or of the number of different individuals enrolled at any given time or over the 
period of a program year. Information that was on hand seemed to have been 
collected to comply with state services reimbursement and related reporting 
requirements. IThe only information available was that obtained through our 
Universe Surv^ey, which gathered enough data for us to generate a sampling frame. 

Designing a probability sample, essential to a rigorous evaluation, proved 
straightforward. But securing a sufficient number of programs to implement the 
design was another matter; that required us to be very flexible and persuasive. 
Initially, letters urging cooperation and participation were sent to selected programs 
by various agencies. Letters from state departments of education proved critically 
important in inducing participation by several programs. We followed up those 
letters with detailed information about the study, its requirements, and the incentives 
available to facilitate cooperation. We then contacted each program by telephone 
and began what often was a long and complicated process of negotiation. 

Our goal was to enlist participation of 150 local programs. When data 
collection began, 141 programs had agreed to participate: 114 from the initially 
selected set of 150; 25 first order replacements; and 2 replacements of replacements. 
After ten months of data collection, 2 of the originally selected 150 had terminated 
operations, 3 had formally withdrawn from the evaluation, and another 5 had failed 
to submit data. Within the first 6 months of the evaluation, 16 percent of program 
directors trained in the requirements of the study had departed, sometimes because 
their positions had been abolished. Additional barriers overcome included resistance 
by most program directors to random selection of instructional delivery sites and of 
clients. 

The evaluation plan placed a heavy data collection burden on participating 
programs. That was necessary, however, because we felt that the most feasible and 
economic way of obtaining information on an ongoing basis over the 30-month data 
collection period was to use local program staff. To ease the burden as mudi as 
possible, we offered incentives and allowed substantial flexibility as circumstances 
warranted. For instance, incentives included reimbursement for extra staff hours 
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expended on data collection, encouragement from state departments of education, 
and psychological rewards (e.g., publicity and supporting letters from education 
agencies). Among situations requiring flexibility was the testing of clients for literacy 
achievement. We offered free CASAS tests to programs, along with training and 
scoring services. Even so, that was not acceptable to many programs. In those cases, 
we agreed use the TABE, ABLE, and BEST tests, as appropriate, and as a last resort, 
we let them participate in the study without providing us with achievement test 
scores (see chapter 3). 

(Available on request is a paper that explores in detail the foregoing 
challenges, among others, and how they were addressed by evaluation staff.^) 



Organization of This Report 

This report has five chapters. Following this introductory chapter, the report 
addresses the following major questions, among others: 

• How many persons enroll in ABE, ASE, and ESL? How many are new 
clients? How long do they persist in the program? What factors 
contribute to client persistence? (Chapter 2.) 

o What are client literacy outcomes as measured by standardized 
achievement tests? What is the relationship between hours of 
instruction and other measured variables to learning outcomes? 
(Chapter 3.) 

• What is the evidence that participation in the program results in 
employment and other meaningful outcomes? What is the relationship 
between hours of instruction and other measured variables to their 
outcomes? (Chapter 4.) 

• How well is the overall program meeting target population 
needs/demand nationwide? To what extent is the program reaching the 
major segments of its target population? Is program capacity adequate? 
What would it take to significantly impact on client needs? (Chapter 5.) 



^ "Description of Problems and Issues Encountered During the National Evaluation of Adult 
Education Programs." Address requests to Development Associates, Inc., 1730 North Lynn Street, 
Arlington, VA 22209-2023. 



Exhibit !•! 

Primary Sources of Information for This Report 



• The Universe Survey surveyed all federally supported adult education instructional 
service providers. Conducted in the fall of 1990, it obtained data from 2,619 (93 percent) of 
the local programs receiving federal basic state grants funds in the program yecir ending 
June 30, 1990. 

• The Comprehensive Program Profile sought more detailed information about program 
structure, instruction, and operatioris than, the information obtained in the Universe 
Survey. A Program Profile was obtained from 131 of the 139 local programs that agreed 
to participate in the longitudinal phase of the evaluation. For programs that provided 
client-level data from more than one instructional site, we obtained data pertaining to 
instructional variables from site directors, and we xised site-level data rather than 
program-level data in the analyses. 

• The Client Intake Record: Part A, which was completed for each sampled client, 
provides basic demographic information on the client as weU as program information, 
such as placement level, scheduled start date, and local intake procedures used for the 
client. Program staff completed the form from program intake records. Analyses in this 
report are based on data for 22,548 clients from 116 local programs. 

• The Client Intake Record: Part B, which was to be completed for all sampled clients who 
attended at least one instructional session, collected more detailed information-on client 
characteristics, including receipt of public assistance, living arrangements, and 
employment status-than Part A contained. Part B also asked clients to rank, by 
importance, 14 reasons for taking adult education instruction. Spariish versions of the 
form were provided as needed. This report draws on data from 13,845 clients in 108 local 
programs. 

• The Client Update Record provides instructional and attendance data and was completed 
at 5- to 8-week intervals by local program staff for each client who received instruction 
during the reporting period for up to 18 months from the time of entry into the program. 
Analyses in this report are based on data from 18,461 clients in 110 local programs. 

• The Client Test Record provides scores on tests of basic skills given near the time of 
enrollment and after varying numbers of hours of instruction. Analyses are based on 
pretest scores from 8381 clients in 88 local programs, and posttest scores from 1,919 
clients in 65 programs across the United States. 

• The Telephone Follow-up Survey provides information about the quality of the 
instruction, reasons for termination, and the results of instruction from a subsample of 
clients 6 months after they left the program. Data from 5,401 clients from 109 local 
programs are used in this report. 
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Chapter 2 

PATTERNS OF ENROLLMENT AND PERSISTENCE 

The central purpose of the National Evaluation of Adult Education Programs 
was to evaluate the potential for programs funded through the Adult Education Act 
to reduce significantly deficits in the adult population with respect to literacy, English 
proficiency, and secondary education. Accomplishing that purpose required 
information on (1) the number of clients served and the amoimt of services they 
received; (2) the extent to which those clients benefitted from the program services; 
and (3) the relationship between the number of clients benefitting from the program 
and type of benefits received and the extent of the target population's need and 
demand for the program's educational services. This chapter provides information 
on the first of those topics. Essentially, we foimd that during the period of the study 
(1991-1993): 

• the program served between 2.6 and 3.2 million clients annually, with 
about 2 million of them newly enrolled; 

• about 46 percent of clients were enrolled in ESL, 30 percent in ASE, and 
24 percent in ABE; 

• about 15 percent of clients who enroll in the program never actually 
receive instruction; 

• the median number of hours of instruction for clients who received 1 
hour of instruction or more was 58 hours; 

• the median number of hours of instruction received by clients enrolled 
in ESL was about 3 times that of those enrolled in ABE (113 and 35 
hours, respectively) and 4 times that of those enrolled in ASE (28 hours); 
and 

• about 76 percent of the total number of instructional hours received was 
by clients in ESL, about 13 percent in ASE, and 11 percent in ABE. 

More detail on some of the topics addressed in this chapter, including tables 
providing estimates of client persistence and analyses of possible predictors of client 
persistence, can be found in ihe study's third interim report. 



Outreach and Recruitment Activities 



The Adult Educati.on Act specifies that 

adult education means services or instruction below the college level for 
adults: (a) who are not enrolled in secondary school; (b) who lack 
sufficient mastery of basic educational skills to enable them to function 
effectively in society or who do not have a certificate of graduation from 
a school providing secondary education and who have not achieved an 
equivalent level of instruction; (c) who are not currently required to be 
enrolled in school; and (d) whose lack of mastery of basic skills results 
in an inability to speak, read, or write the English language, which 
constitutes a substantial impairment of their ability to get or retain 
employment commensurate with their real ability and thus are in need 
of programs to help eliminate such inability. 

Among adult education professionals, an area of long-standing concern is how 
to reach those persons with the greatest need. For many years researchers in the 
field have been concerned about the extent to which programs are serving people 
who are the easiest to reach, rather than those who might be considered "most in 
need" (Mezirow, Darkenwald, and Knox 1975; Beder 1991). As we discuss in chapter 
5, we conclude that the program is serving a population tiiat needs adult education 
services, that there is an overall youth bias in the program, which is appropriate 
given the stated purposes of the act, but that there is a bias, particularly in the West, 
toward clients who need ESL services which may work to the detriment of other 
segments of the population. Here we briefly describe the techniques that programs 
use to reach potential clients and the ways clients report they learned abotit the 
programs. 

The proportions of programs using various recruitment techniques are shown 
in exhibit 2.1. The proportion of programs using any one of the techniques ranges 
from 34 percent to 92 percent. Three techniques used by more than two-thirds of the 
programs are announcements in mass media, recommendations by current clients, 
and contacts with supervisors or counselors. Almost all (92 percent) programs that 
predominantly serve ABE clients report that they contact potential clients' supervisors 
or counselors. Programs that serve predominantly ESL or ASE clients are less likely 
(75 and 71 percent, respectively) to use this approach and somewhat more likely to 
use announcements in mass media. In general, programs predominantly serving ESL 
clients are less likely to use a wide variety of recruiting techniques than programs 
predominantly serving ABE or ASE clients. 
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Exhibit 2.1 

Proportion of Programs Using Various Recruitment Techniques 
By Predominant Instructional Component^ 
(N = 131) 



Recruiting Technique 


Percent of Programs 


ESL 


ABE 


ASE 


Overall 


Announcements in mass media 


85 % 




66 % 


75% 


70 % 


Recommendations by current clients 


76 




61 


74 


66 


Contacts with supervisors or counselors 


75 




92 


71 


77 


Referrals from other agencies 


74 




73 


54 


63 


Local residents used as recruitment aides 


68 




52 


52 


56 


Recruitment by cosponsoring groups 


49 




76 


62 


68 


Fliers, posters, mailings 


47 




63 


58 


58 


Staff member assigned to recruitment 


44 




57 


61 


57 


Organized recruitment by current clients 


37 




64 


51 


53 


Other^ 


11 




9 


31 


17 



^ Programs were defined as predominantly of one type if a plurality (but not 
necessarily a majority) of their clients were enrolled in that component. The 
distribution of programs by predominant instructional component is 21 percent ESL, 
48 percent ABE, and 31 percent ASE. 



^ Respondents were provided the opportimity to specify two "other" ways they use to 
recruit potential adult education clients. 

Clients' responses about how they learned about the program are presented in 
exhibit 2.2. Most learn about the program from friends or family members. Clients 
are least likely to learn about the program through the mass media, which, according 
to program directors, is the second most frequently used recruitment method. More 
specifically, clients report that they learned about the adult education program from 
the newspaper (5 percent), television (3 percent), or radio (2 percent). V\T:iile these 
findings raise questions about the wisdom of program expenditures on mass media 
advertising, we have no information about how clients' friends and family learned 
about the program. Particularly for ESL and ABE clients, recruitment may be a 
multistep process, and the mass media may be an effective way of reaching potential 
clients' families and friends. 
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Exhibit 2,2 

Percent of Clients by How They Learned About the Program 

(N = 13^35) 




Number of Clients Being Served 

Estimating the number of adult education clients the program serves in a year 
is more complex than estimating the number served at any one point in time. 
Individual clients may enroll, intermpt or discontinue, and reem*oll in programs at 
various times during any given year. Furthermore, federal reporting requirements 
define an adult education client as someone who has received 12 or more hours of 
instruction. As a result, careful recordkeeping by local programs is required for 
state-level information to be accurate, and the extent to which local programs are able 
to track individual students varies considerably. 

The study plan involved selecting a nationally representative sample of new 
clients, using a multistage selection procedure. This sample resulted in a national 
estimate of 2,016,288 new clients entering adult education programs over the 12- 
month period between April 22, 1991, and April 21, 1992. 

To develop national estimates o^ the total number of clients that received one 
or more hours of instruction over a 12-month period, we took several additional 
steps. It was necessary to take into account clients who were systematically excluded 
from the sampling frame (e.g., those in prisons, hospitals, and homes for the 
retarded), clients who were in programs and instructional sites that opened after the 
sample was drawn, clients who were inappropriately coimted or not coimted at all 
because of program reporting errors, and clients who entered the program in 
previous years and were receiving instruction when our survey began. 
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After adjusting the data for the circumstances just described, we estimate that 
between 2.6 and 3.2 million adult education clients received 1 or more hours of 
instruction during the 12 months between April 22, 1991 and April 21, 1992. Exhibit 
2.3 presents the basis for these estim^':ttes, beginning with the number of new clients. 
Of the 2.6-3.2 million clients who received 1 or more hours of instruction, 
approximately 84 percent or 2.2-2.7 million received 12 hours or more of instruction. 

Some 46 percent of new cUents enrolling in adult education during this period 
enrolled in ESL classes; ASE accoimts for 30 percent of new clients; and ABE accounts 
for the remaining 24 percent. Of clients who received one hour of instruction or 
more: 

• 51 percent are in ESL, 

• 30 percent in ASE, and 

• 19 percent in ABE. 

Over the past 15 years the proportion of ESL clients has been increasing while the 
proportion of clients in the other two components has decreased. In 1979, about 19 
percent of the program's clients were enrolled in ESL, 3^ percent in ASE, and 44 
percent in ABE (Young et al. 1980). 

The distribution of all clients by census region shows that 40 percent live in 
the West, 30 percent in the South, 19 percent in the North Central region, and 11 
percent in the Northeast. As exhibit 2.4 shows, 82 percent of clients in the western 
states enrolled in ESL, while less than 25 percent of clients in each of the other three 
census regions enrolled in ESL. 
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Exhibit 2.3 

High and Low Estimates of the Number of Clients Receiving 
1 or More Hours of Instruction Over a 12-Month Period 





High Estimate 


Low Estimate 


1. 


Number of newly eru"olled clients based on study sample 


2,016,288 


2,016,288 


2. Possible reporting problems (± 10%) 


+201,629 


-201,629 


3. 


Number of clients who enrolled but never received 
instruction (-15%) 


-324,106 


-265,178 


4. 


Number of clients excluded from the sampling frame: 








a. clients in prisons, hospitals, and other institutions 
(+12 % of clients with instruction) 


+227,257 


+185,938 




b. clients in programs that began after the sample was 
drawn^ 


+6,000 


+6,000 




c. clients in mstructional delivery sites that opened after 
the sample was drawn^ 


+28,000 


+28,000 


5. 


Number of clients already being served when the study 
year began^: 








a. clients who entered the program 1 to 12 months earlier 
and were still active at the start of the intake year 
(+35% of new clients, i.e., steps 1-4) 


+755,895 


+618,460 




b. clients who entered the program 13-24 months earlier 
and were stiill active at the start of the intake year 
(+8.9% of new clients, i.e., steps 1-4) 


+192,104 


+157,176 




c. clients who entered the program 25 or more months 
earlier and were still active at the start of the intake 
year (+3% of new clients, i.e., steps 1-4) 


+64,754 


+52,981 


Total number of clients served 


3,166,226 


2,599,631 



^ Number of clients in excluded programs (4.b) is based on sampled programs that closed prior 
to data collection and on discussioixs with state directors indicating ttiat approximately equal 
numbers of small programs open and close each year. 



^ Number of clients in excluded sites (4.c) is based on contact with program directors regarding 
sites that opened during the study period. 

^ Numbers of clients already being served (5.a, 5.b, 5.c) are based on extrapolations from 18 
months of attendance data (see the second interim report for discu-^sion of procedures used). 
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Exhibit 2.4 

Enrollment by Instructional Component and Region of Country 

(N = 22,548) 



Instructional 
Component 


Percent of Clients 


Northeast 


North Central 


South 


West 


Overall 


ESL 


24% 


21 % 


20 % 


82 % 


46% 


ABE 


35 


37 


39 


4 


24 


ASE 


41 


42 


41 


14 


30 


Overall 


100 % 


100 % 


100 % 


100 % 


100 % 



Patterns of Client Enrollment 

Despite program descriptions that emphasize an "open-entry, open-exit" 
instructional design, the enrollment and attendance pattern for most adults is 
consistent with the traditional fall entry/summer exit pattern of elementary and 
secondary schools or the September-January-March community college enrollment 
cycle. September and January are the months of greatest intake, with August and 
March providing the next highest numbers. Estimates of the proportion of intakes by 
month and instructional component are presented in exhibit 2.5. 
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Exhibit 2.5 

Proportion of New Clients by Month of Enrollment 

(N = 22,548) 



Percent of Clients 


Month 


ESL 


ABE 


ASE 


Overall 


January 


11 % 


10 % 


14 % 


12 % 


February 


8 


6 


10 


8 


March 


13 


5 


7 


9 


April 


6 


6 


6 


6 


May 


5 


11 


7 


7 


June 


4 


7 


5 


5 


July 


4 


9 


5 


5 


August 


13 


6 


6 


9 


September 


24 


21 


21 


22 


October 


7 


10 


10 


9 


November 


3 


6 


6 


4 


December 


2 


3 


3 


3 


Total 


100% 


100 % 


100 % 


100 % 



About 15 percent of the clients who enroll for adult education services do not 
actually begin instruction. Exhibit 2.6 shows the proportion of clients who began 
instruction by month of enrollment. About 92 percent of clients who enrolled in 
March started instruction, while about 80 percent of May enroUees started instruction. 
Some 48 percent of ESL, 59 percent of ABE, and 55 percent of ASE local adult 
education programs take a 2- or 3-month summer break, and clients who enroll in 
programs in April and May may decide not to begin instruction because they know 
instruction will end within the next 6 to 8 weeks. 
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Exhibit 2.6 

Instructional Status of Clients by Month of Enrollment 

(N = 18,461) 



500,000 
400,000 
300,000 
200,000 
100,000 



m 



Percent of Intakes 
Who Began Instruction 




Apr May 



Jun 



Jul 



Aug Sep Oct Nov 
Month of Intakes 



Jan 



Instruction I 



I Never 
Began 



As with other aspects of the adult education program, enrollment patterns 
vary by region of the United States. As exhibit 2.7 shows, programs in the Northeast 
enroll nearly 40 percent of their annual new clients in September, while programs in 
the South enroll clients more consistently throughout the year. 



Exhibit 2.7 
Variations in Enrollment Patterns 
By Region and Month of Enrollment 
{N = 22348) 



Northeast 



North Central 



40 

S 

J5 30 

2 
« 
Z 

f 




S 30 



^ 20 



g 10 



Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar 
Month of Intake 



I I. I.. 

Apr May Jwi Jul Aug Sep Oct Nov Dec Jan Feb Mar 
Month ol Intake 



S 

S 30 



1.0 



South 



Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar 
Month of Intake 




Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb M*r 
Month of Intake 



17 



ERIC 



-J 



Besides regional variations in monthly patterns of enrollment and variations by 
type of instructional program, the time of day clients choose to receive instruction 
varies regionally as well. As shown in exhibit 2.8, a considerably larger proportion of 
clients in the West are enrolled in daytime classes only. The West also has the 
highest proportion of clients who were enrolled in ESL (82) and the highest 
proportion employed at the time of intake (46 percent). Apparently a large 
proportion of tiie clients in the West work at night and attend classes during the day. 
In contrast, in the Northeast only 29 percent of beginning ESL clients attend classes 
during the day only. The regional findings may reflect the tj^es of jobs held by 
persons with limited English language skills in the two parts of the coimtry. 

Exhibit 2.8 

Proportion of Clients Enrolling in Daytime Classes Only, 
By Region and Initial Level of Instruction 
(N = 15,701) 



Instructional 
Component 


Percent of Clients in Daytime Classes Ordy 


Northeast 


North Central 


South 


West 


ESL-beginning 


29 % 


63% 


57% 


68 % 


ESL-advanced 




62 


56 


57 


ABE-beginiiing 


55 


41 


52 


61 


ABE-intennediate 


66 


53 


53 


56 


ASE 


41 


51 


46 


52 


Overall 


47 


52 


50 


63 



Patterns of Client Persistence 

Throughout this report, we use the term persistence to identify a client's total 
amount of participation in the adult education program, but participation in adult 
education programs can be measured in various ways. In this report we use the 
following measures: 

• Total weeks of enrollment. The number of weeks between the start 
and the end of services, regardless of the amount of services received 
during that period. 
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• Total hours of instruction. The number of hours a client actually 
attends classes or receives some other instructional service. 

• Total weeks of instruction. The number of weeks during which a 
client actually attends class. 

Although there is a relatively high correlation^ among the three measures, 
they are conceptually and operationally distinct, and results differ somewhat, 
depending on the measure used. For this report, most analyses are in terms of total 
weeks of enrollment or total hours of instruction. 

With respect to median hours of instruction for clients who received 1 hour of 
instruction or more, as exhibit 2.9 shows: 

• The median for all three components is 58 hours. ESL clients had the 
highest median hours of instruction (113), followed by ABE clients (35) 
and ASE clients (28). 

• The West had the highest median hours of instruction with 107. This 
finding is directly related to the high proportion of ESL participants in 
the West (82 percent). The Northeast had the next highest median 
hours of instruction with 50. 

• The median for ESL clients was higher than for ABE or ASE clients, 
especially in the West, which had a median of 136 hours. ASE clients in 
the South had the lowest median hours of instruction (24) across all 
components and all census regions. 

In terms of total hours of instruction received, clients in ESL classes account 
for approximately 76 percent; ASE, 13 percent; and ABE, 11 percent. That is, over 
three-fourths of the client seat hours supported by the Adult Education Act are 
provided to clients enrolled in the ESL component. 



^ The correlation across all components between total weeks of enrollment and total weeks of 
instruction is .91 and between total weeks of instruction and total hours of instruction is .85. A 
correlation matrix showing the relationship among these variables overall and by component is 
presented in the third interim report. 
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Exhibit 2.9 
Median Hours of Instruction of Clients 
Who Received 1 Hour or More 
(N = 15,870) 



Region 


Hours 


ESL 


ABE 


ASH 


Overall 


North Central 


57 


25 


34 


34 


Northeast 


77 


58 


33 


50 


South 


62 


30 


24 


30 


West 


136 


36 


25 


107 


Total U.S. 


113 


35 


28 


58 



The persistence rates for those clients who received at least 1 hour of 
instruction are displayed in exhibit 2.10. As this graph shows, rates for ABE and ASE 
are nearly identical, with ABE clients continuing at a slightly higher rate than their 
ASE counterparts. The curves demonstrate that over time fewer and fewer clients 
attend. 

During the first year, the rate of decline for ESL is considerably different from 
ABE or ASE. Whether calculated in weeks or hours, ESL clients receive substantially 
more instruction during the first 12 months following enrollment than do clients 
eru-olled in ABE or ASE. Shortly into the second year of instruction, the number of 
ESL clients continuing to receive instruction declines fairly sharply, and by the end of 
the 18-month follow-up period, the rates for the three components have nearly 
converged. 
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Exhibit 2.10 

Persistence Rates in Weeks for Clients Who Received 
1 Hour or More of Instruction by Instructional Component 




6 12 18 24 30 36 42 48 54 60 66 72 78 
Weeks EmoUed 



Exhibit 2.11 details the average number of weeks of enrollment in each of the 
three program components by month of erirollment, and overall. Clients entering the 
program in Augxist had the highest average number of weeks of enrollment in all 
three instructional components; ESL clients entering in August had the highest 
average number of weeks of enrollment (40), followed by ABE clients (28) and ASE 
clients (23). 

Exhibit 2.11 further illustrates that clients in ESL stay enrolled in programs 
longer than ABE or ASE clients. Overall, ESL clients averaged 30 weeks of 
enrollment, compared with 20 weeks for ABE and 17 weeks for ASE clients. Across 
the three components, the average is 24 weeks for clients to be enrolled. When 
assessed in terms of census region, we find that there is much less variation in the 
length of erurollment in the South, regardless of the instructional component, than in 
other regions, and that the West had the highest average number of weeks of 
enrollment across all three components (see appendix D, exhibit D.9). 

Of all new enrollees, 38 percent enrolled for less than five weeks. Although 
many clients who do not complete their first month receive no instruction at all, 
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about 23 percent of all clients begin receiving instruction but leave within 5 weeks of 
when they enrolled. By component, the proportion of new enrollees who begin 
receiving instruction but leave within their first 5 weeks is about 26 percent for ABE, 
30 percent for ASE, and 19 percent for ESL. 

Exhibit 2.11 

For ESL, ABE, ASE: Mean Number of Weeks Clients Remained Enrolled, by 

Enrollment Month 



component 




Jan 


Feb 


Mar 


Apr 


May 


Jun 


Jul 


Aug 


Sep 


Oct 


Nov 


Dec 


Overall 


ESL 


28 


28 


37 


37 


22 


29 


26 


40 


23 


27 


22 


32 


30 


ABE 


19 


19 


15 


16 


18 


21 


21 


28 


22 


19 


17 


16 


20 


ASE 


15 


15 


13 


15 


14 


14 


14 


23 


20 


18 


18 


18 


17 



Note: Number of weeks of enrollment is defined as the number of weeks that elapsed between initial instruction and 
final departure of the client from the program. In Appendix D, exhibit D.9 shov/s data above by census region. 



Additional detail on client flows is presented in Appendix D. The first four 
tables in the appendix show the proportion of all new clients exiting the program at 
specified numbers of months after they ervrolled. These tables show, for example, 
that regardless of when they enroll, clients are most likely to leave the program 
during their first month. Of the clients who enter the program in August or 
September, 29 percent leave the program before completing 1 month. 

Also included in Appendix D are tables which provide estimates of the 
persistence of all new clients in terms of a hypothetical group of 100,000 new 
enrollees. The data in the persistence tables show the same trends as described 
above. The declirung rate of departure indicates that the initial month of contact is 
crucial for lon^-term persistence. On average, clients complete about 5 months of 
instruction between when they enroll and when they leave the progreon. However, if 
clients make it into their second month, they are likely to complete another 7 months, 
for a total of 9 months of instruction. 

Exhibit 2.12 presents estimated fluctuations in the total number of active 
clients over a 12-month period. The number of clients reaches its low point in July 
and peaks in September/October and March. 
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Exhibit 2.12 

High and Low Estimates of Total Number of Active Clients by Month 




Note. The estimates involve alternative assumptions about implementation of the 
original sample design and establishment of new programs. 

Program Factors Contributing to Client Persistence 



As we discuss in greater detail in chapter 3, attending class for many hours is 
not necessarily related to achieving meaningful learning gains. In chapter 3 of the 
third interim report, we presented the results of analyses that identified major 
predictors of receiving a relatively large number of instructional hours.^ Predictors 
and causes, of course, are not necessarily the same, and it is important to keep this 
distinction firmly in mind when reviewing the findings presented below. 

We fotmd that several variables within the control of local programs are 
important predictors of the number of hours of instruction that clients receive. 
Across all three components, the following program factors were the strongest 
predictors of persistence: 

• the presence of support services that clients actually use; 

• the receipt of instruction during the day as opposed to evening hours; 
and 



• the type of learning environment in which the clients participate. 

Use of support services is a strong predictor for all instructional components. 
The presence of such services may well explain why some clients can sustain their 
participation in adult education and others carmot. We found that about 25 percent 
of clients use at least one type of support service. The support services most 
frequently used were counseling (16 percent), financial assistance (7 percent), and 
transportation (7 percent). Clients in programs that provided five types of services or 
more received, on average, 115 hours (and 19 weeks) of instruction, whereas clients 



^ Logistic models were developed to identify predictors of persistent attendance for clients in 
each of the program's three instructional components. In these models, clients who took 12 hours or 
more of instruction in the first three attendance quartiles were compared with clients who were in the 
fourth quartile (the persisters). 
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in programs that provided four types of support services or less received, on average, 
60 hours (and 17 weeks) of instruction. 

We also looked at the degree to which services are integrated (i.e., how well 
the educational services are coordinated with services provided by other agencies, 
and the breadth of the services). Integration of services was classified as high, 
medium, or low. The relationship between our measure of integration of services 
and client persistence was most clear for the ABE instructional component. ABE 
clients are more likely to persist in programs that score well on integration of 
services. 

The second particularly strong program factor — receiving instruction during 
the day — is predictive of persistence for all three instructional components. 
Regardless of component, clients who attend classes only during the daytime are the 
most likely to persist, and those who attend only at night are the least likely. While 
the time of day that instruction is received is highly predictive of sustained program 
attendance, it almost certainly is not the primary cause. Rather, it is likely that clients 
who are free to attend classes during the day can attend classes for more hours at a 
time (morning and afternoon class sessions often last longer than those at night) and 
can attend more frequently. Also, ability to attend classes during the day generally is 
a function of family or employment characteristics, which may be the real explanation 
for our results. While such factors were included in our analyses, it is reasonable to 
believe that their importance was hidden because no one or two such reasons were 
predominant. 

How the third particularly strong factor — a client's learning environment — is 
related to persistence differs by instructional component. ESL clients whose 
instruction includes independent study or participation in a learning lab environment 
are more likely to persist than those whose instruction is only classroom based. 
Participation in a learning lab rather than only classroom-based instruction is also 
predictive of persistence for ASE clients, but having a program that includes 
independent study is not. ABE clients whose instruction is provided only in a 
teacher-based classroom are more likely to persist than those whose instruction also 
includes independent study. It may be that ABE clients need the structure and 
nurture provided by teacher-based classrooms and are not yet ready for the 
combination of classroom and independent study or lab settings in which ASE and 
ESL clients are most likely to thrive. 

Three additional program variables predict persistence for two of the three 
instructional components. Class size, for example, is a predictor of persistence for 
ESL and ABE clients, but is not for clients in ASE. ESL clients are more likely to 
persist when enrolled in large classes, whereas ABE students are more likely to 
persist in classes of medium to large size (more than 10 clients). Our finding, 
particularly with respect to ESL, may be more reflective of common than exemplary 
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practice. ESL students tend to persist longer than those in ABE or ASE, and ESL 
programs tend to be at or above capacity enrollment and to have large classes, 
whether they are particularly predictive of learning outcomes or not. 

Having at least one full-time administrator and one full-time instructional staff 
member associated with the program is a strong predictor of persistence for ESL and 
ASE clients. We suspect that having such a staff configuration increases the quality 
of instructional staff training, supervision, and support, which we assume should also 
be the case for ABE. 

The cost of instruction was also foimd to be related to persistence, but in 
somewhat surprising ways. To estimate the cost of providing instruction the 
evaluation collected information by mail survey from all participating programs on 
the cost of service provision. In addition, there was a special cost study consisting of 
case studies of 12 programs selected to be nationally representative in terms of 
number of clients served, size of annual budget, type of sponsoring institution, and 
geographic region. On the basis of the case studies and survey data, we estimate that 
cash cost per hour of instruction is $4.57.^ For the purposes of the analyses of 
persistence and learning gains presented in this report, we categorized programs in 
terms of cost as "average," "above average/' and "below average," on the basis of one 
standard deviation above or below the mean hourly cost (a = $2.80). 

Spending more per hour of instruction is not positively related to persistence. 
Client cost per seat hour is not a predictor of persistence for ESL, although as will be 
seen in chapter 3, it is predictive of learning gains. Similarly, although low cost per 
seat hour is predictive of persistence for ASE clients, high cost per hour is predictive 
of instructional gains. For ABE, average costs are predictive of persistence, but 
persistence per se is not predictive of learning gains. We suspect these findings with 
respect to cost per hour are a reflection of instructional environment and staff 
configurations. What seems to be important is how money is spent, not simply that 
more is made available. 

Other Predictors of Client Persistence 

In addition to those program factors, two personal characteristics predict client 
persistence: race /ethnicity and age. Race /ethnicity predicts persistence in all three 



^ Across the 12 case study programs the cash cost per seat hour ranged from $0.82 to $9.80, and the 
total (cash plus noncash) cost ranged from $1.61 to $9.84. The mean cash cost of an hour of instruction in 
the 12 case study sites was $4.57, with a standard deviation (o) of $2.80. Using the responses from the mail 
survey of local program directors plus information from our national sample of clients we found that client 
cost per hour of instruction was $4.48. For the tl^«'ee instructional components, on the basis of the more 
detailed case study data, we calculate the mean costs per hour as: ESL = $4.28 (a = $3.84), ABE = $6.11 (a 
=: $3.47), and ASE = $5.12 (a = $2.69). Copies of the special cost study report may be obtained by 
contacting the Department of Education or the study authors at Development Associates, Inc. 
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program components. In ABE and ASE we found that Asians /Pacific Islanders were 
more likely to persist than non-Hispanic whites. In addition, Hispanics were more 
likely to persist in ABE than non-Hispaiuc whites. In the ESL component, we found 
that non-Hispanic whites were more likely to persist than Hispanics. Also, we found 
that age was not a substantial predictor of persistence for clients enrolled in ASE or 
ESL; but ABE clients over the age of 30 were ir :e likely to persist than yoimger 
clients. 

Throughout all instructional components, the following personal variables 
were not predictors of persistence: sex, marital status, welfare status, prior education, 
whether enrollment was required or volxintary, and the client's primary purpose for 
enrolling. 

Summary and Conclusions 

In this chapter, we note that the outreach and recruitment techniques used by 
more than two-thirds of the programs are announcements in mass media, 
recommendations by current clients, and contacts with supeivisors and counselors. 
However, most clients leam about the program directly from friends or family 
members. 

We find that most clients enroll in September and January. Enrollment, 
however, does not guarantee that clients will actually attend class or begin 
instruction. The proportion of enroUees actually beginning instruction varies 
somewhat by month. For example, 92 percent of clients who enrolled in March 
received 1 or more hours of instruction, while only 79 percent of clients who enrolled 
in April or May began instruction. 

Enrollment patterns also vary by U.S. census region. Programs in the 
Northeast enroll nearly 40 percent of their new clients in September, whereas 
programs in the South enroll clients at a fairly consistent rate throughout tl\e year. 
The time of day that classes are scheduled also varies by region. Clients are more 
likely to be enrolled in daytime-only classes in the West than in any other region. 

Regarding patterns of client persistence, we foimd that ESL clients stay in their 
programs longer and receive more hours of instruction than ABE or ASE clients. ESL 
clients average 30 weeks of enrollment, compared with 20 weeks for ABE and 17 
weeks for ASE clients. ESL students receive substantially more instruction during the 
first 12 months of enrollment than students in ABE or ASE. 

To identify predictors, we used logistic regression to compare data on clients 
in the first three attendance quartiles to clients who were in the fourth quartile. We 
found that the strongest predictors are factors over which local programs have some 
control — program structure and design — rather than personal client characteristics. 
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Our study revealed that the strongest predictors of persistence across all components 
are: 



• the presence of support ser\dces that clients actually use; 

• the receipt of instruction during the day as opposed to evening hours; 
and 

• the type of learning environment in which clients participate (but the 
type of environment that was most predictive was not the same for all 
three instructional components). 

There is an understandable tendency to think of student persistence as an 
appropriate measure of program effectiveness. The assumption is that the longer 
clients spend in adult education programs, the more likely they are to accomplish 
their objectives or the objectives the program has set for ihem. As we discuss in the 
context of analyses of client outcomes presented in chapters 3 and 4, however, the 
relationship between client retention and program effectiveness is complex. For some 
clients it is appropriate to equate time spent in adult education with progress toward 
learning gains or other desired objectives, but for other clients it is not. 
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Chapter 3 

CLIENT LITERACY OUTCOMES AS MEASURED BY 
STANDARDIZED ACHIEVEMENT TESTS 

Adults enroll in instructional programs for many reasons, but according to the 
National Literacy Act, federally supported local programs are intended to enable 
participating adults to: 

1. acquire the basic educational skills necessary for literate functioning; 

2. benefit from job training and retraining programs and obtain and retain 
productive employment; and 

3. continue their education to at least the level of completion of secondary 
school. 

This chapter presents findings on client learning gains pertinent to the first of 
these three objectives. Specifically, this chapter is concerned with literacy outcomes 
measured by the reading subtests of the CASAS (California Adult Student 
Assessment System) and the TABE (Test of Adult Basic Education). While the 
learning gains of adult students participating in the federal Adult Education Program 
are described here, the main emphasis is on identifying factors that directly 
contribute to these gains within each of the three program components (ESL, ABE, 
and ASE).^ Findings on other indicators of client learning-employment outcomes, 
and other client benefits of program participation relevant to the second and third 
objectives of the Act-are presented in chapter 4. 

A primary aim of the leaming gains analysis was to estimate the effects of 
instruction by determining whether student achievement was related to attendance in 
adult education (see Study Objective 7 in Appendix A of the report). Because a 
quasi-experimental design was unfeasible, a regression analysis framework was used 
to assess the effects of instruction and other factors on student achievement. Under 
the regression framework employed in the leaming gains analysis, posttest 
performance was the outcome of interest and the regression coefficient associated 
with total hours of instruction represented the overall effect of adult education 
instruction. Indicators of student attendance were therefore viewed as pivotal to the 



5 Analyses of ESL leaming outcomes were based on CASAS data since programs overwhelmingly 
used the CASAS in testing ESL students. Similarly, analyses of ABE and ASE leaming outcomes were 
based on TABE scores since programs most often used the TABE in testing ABE and ASE students. 
The CASAS is a criterion-referenced test whereas the TABE is a norm-referenced test. 
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analysis of client literacy outcomes since the logic of the evaluation was that "learning 
gains" should be a positive function of student "persistence" in the program. 

The Availability and Quality of Test Data 

Based on intake records received from 101 programs agreeing to provide us 
witli test data,^ we estimated that matched test scores (i.e., pretest and posttest 
scores for the same person from the same test) were potentially available for 19,796 
clients. All 101 of these programs actually did provide pretest data, but only for 57 
percent of the potential base number of clients (i.e., 11,354 out of 19,796). Thus, the 
actual base of test scores was roughly half of what we expected. The achievement 
test data base was further reduced, severely so, in attempting to obtain posttest 
scores for those clients who had been pretested. While matched pretest and posttest 
scores were provided by 74 percent of the programs in the test sample (i.e., 75 of 101 
programs), the number of matched scores obtained for clients on all tests constituted 
only 12 percent of the potential total number of clients originally expected (i.e., 2,333 
out of 19,796). Almost all of the matched test scores were from the CAS AS or the 
TABE (n = 2,315). For this reason, the learning gains analyses were restricted to these 
two tests. 

Using several selection criteria, including known client placement and test 
content validity (i.e., the test administered must adequately measure the local 
program's curriculum as judged by the degree to which the content of the test 
matches the content of the curriculum), five potential analysis groups were formed 
from the 2,315 clients with matched CASAS and TABE scores. The selection criteria 
used in forming the five groups, particularly the content validity criterion, reduced 
the test sample from 2,315 to 1,642 clients. The five potential analysis groups 
constituted 8 percent of the total expected test sample and provided data from the 
two principal tests and all three instructional components of the Adult Education 
Program.'' Two of the five test samples were limited in size and were subsequently 
dropped from further consideration.^ 

The learning gains analyses were conducted with the three remaining test 
groups, consisting of 1,331 clients representing 7 percent of the number of clients for 



^ Local programs were asked to report pretest data using the Intake B Record. Posttest data were 
to be reported on the Progress Record after 70 and 140 hours of instruction. In addition to the CASAS 
and the TABE, programs could also report test scores from the Adult Basic Learning Exam (ABLE) 
and from the Basic English Skills Test (BEST). 

Corisideration of a sixth group — ESL-TABE - was unrealistic because this sample consisted of 
only 10 ESL clients. 

^ The following samples were dropped because of their relatively small size: an ABE-CASAS 
sample of 121 clients and an ASE-CASAS sample of 190 clients. 
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whom test score data potentially cotild have been available. The ESL-CASAS group, 
the largest, comprised 685 clients. The next largest group - ASE-TABE - consisted 
of 454 clients. The smallest group was ABE-TABE, consisting of 192 clients. 

Exploratory analyses revealed a number of problems in the test data, which in 
our opinion, invalidated the scores for over half of the clients in the three final 
analysis groups. Exhibit 3.1 summarizes the validity problems and the extent to 
which they affected these three analysis groups. The proportion of cases affected by 
floor and ceiling effects is indicated by a range statistic in exhibit 3.1 since matched 
scores could be lost by either an inaccurate pretest or an inaccurate posttest. 



Exhibit 3.1 

Summary of Validity Problems and Their Effects on Test Score Attrition 



Validity Problem 


ESLCASAS 


ABE-TABE 


ASE-TABE 


(N - 685) 


(N = 192) 


(N = 454) 


Non-Reading Test 


4% 






Floor Effects 


<1% 


1-2% 


<1% 


Ceiling Effects 


13-18% 


32-33% 


50-65% 


Invalid Pretest 


30% 


3% 


9% 


Total Number of 


n = 336 


n = 81 


n = 300 


Cases Affected 


(49%) 


(42%) 


(66%) 


Final Sample Size 


N = 349 


N = 111 


N = 154 



Scores measuring areas other than reading were not germane to the analysis; 
they were a minor problem and affected only 4 percent of the ESL-CASAS group. 
The presence of "floor effects" in all three samples was another validity problem 
whose impact on the test data was relatively minor. Floor effects are represented by 
chance-level scores on a test (defined in the learning gains study as a raw score of 
less than 30 percent correct) and indicate that the test level administered was too 
difficult for the student; the resulting score simply does not provide an accurate 
measure of the student's level of achievement. Less than 2 percent of the cases in the 
three final analysis groups showed floor effects. 

The opposite problem - "ceiling effects" - had a major impact on the validity 
of test scores, particularly for ASE students administered the TABE. Ceiling effects 
(defined in the study as a raw score in excess of 85 percent correct) generate 
measurement error because the test is too easy and therefore does not provide an 
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accurate assessment of student achievement.^ By far, ceiling effects on the posttest 
(particularly with the TABE) provided tlie single greatest threat to the validity of test 
scores in the learning gains study. For example, despite that more than 90 percent of 
the ASE group was administered the top two levels of the TABE, more than a third 
of the Level D (Difficult) posttest scores and over half of the Level A (Advanced) 
posttest scores were in the ceiling range of the TABE. In contrast, the TABE 
appeared to be somewhat more suited to measuring reading achievement with the 
ABE group, the vast majority of whom took the middle two levels of the test. 
Nevertheless, almost half of the ABE students taking the moderate-difficulty level of 
the TABE (Level M) posttested in the ceiling range. 

The other major validity problem was that of invalid baseline measures as 
reflected by an imreasonable amoimt of instructional time preceding the 
administration of the pretest. This problem had its greatest impact on the ESL- 
CASAS sample where 30 percent of the students received in excess of 50 hours of 
instruction before being given the pretest. The ABE and ASE samples were much 
less affected by the problem of invalid baseline measures despite that the criterion for 
a valid pretest in these two groups was much more conservative (i.e, no more than 7 
hours of instruction prior to the pretest). 

The final vinalysis sample corisisted of 614 clients, or approximately 3 percent 
of the total potential test sample.^^ Appendix E contains a detailed technical 
treatment of methodological issues related to the learning gains analysis, including 
additional information on the quality of the test score data and how these 
considerations affected the determination of the final analysis groups. 

The Learning Gains Analyses 

A weighted analysis strategy was employed (based on a set of revised 
sampling weights) to adjust for non-random subsamples in each of the three program 
components. The reweighting adjustments provide an improved basis for 
generalizing the results of the learning gains analysis to the population of adult 
students participating in the federal Adult Education Program. Although clients in 
the final analysis samples attained a greater number of instructional hours, on 
average, compared to their counterparts in the overall study sample, the weighted 



^ In the exploratory analyses, measurement error produced by ceiling effects invalidated the ASE 
regression model. Only by removing the ceiling effect was it possible to develop a valid model of 
ASE achievement. See Appendix E for further detail. 

^° The final regression analyses are based on data from 614 clients (ESL-CASAS = 349; ABE-TABE 
= 111, and ASE-TABE = 154) from 44 local programs located in 20 states (ESL = 8 states; ABE = 16; 
ASE = 14). 
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data included in the final analyses are generally representative of the program as a 
whole.^^ 

One-sample t-tests were computed to examine the statistical significance of 
gain scores for each of the three program components. The t-test results are 
displayed in exhibit E.5 of Appendix E. In addition, analysis of covariance 
(ANCOVA) was used to examine the statistical significance of differences in adjusted 
posttest means associated with participation in the three principal instructional 
environments (i.e., classroom-only versus lab-only versus classroom plus lab)^^ 
within each program component. Independent variables shown by regression 
analysis to be significant predictors of posttest performance were used as covariates 
in the ANCOVAs. The ANCOVA results are displayed in exhibits E.9 through E.ll 
of Appendix E. 

Separate ordinary least squares multiple regression models were developed to 
identify direct effects iriluencing literacy outcomes in the ESL, ABE, and ASE 
components of the program. Each regression model involved a seven-block 
hierarchical analysis strategy consisting of the following independent variables: 

1. The client's background characteristics (sex, age, race /ethnicity, and 
educational attainment). 

2. An indicator of whether the client was required by his employer or 
anther program (e.g. JOBS) to participate in the program (as opposed to 
enrolling for some personal reason). 

3. The client's pretest reading scale-score. 

4. A set of variables relating to program characteristics (curriculum design, 
client's use of support services, presence of full-time staff, whether the 
majority of teachers were certified or had 3 or more years of experience 
in adult education, and the cost per client seat hour of instruction). 



See exhibits E.l through E.3 in Appendix E for a comparison of the weighted analysis samples 
with the weighted total study samples for each of the three adult education components. 

^2 Environment was measured as a categorical variable in the ANCOVA procedure (i.e., being a 
participant in one of three instructional settings) whereas in the regression analysis, environment was 
measured as a continuous variable (i.e., hours of instruction in a particular environment). The 
rationale for the ANCOVA was to determine whether exposure to a particular instructional setting per 
se (after statistically removing the influence of other factors such as differences in student entering 
ability) resulted in different literacy outcomes, or whether the effect of environment depended on the 
amount of instruction in a particular setting (as examined by the regression analysis). 
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5. 



The mean class size across instructional environments and reporting 
periods. 



6. Measures of instructional intensity and attendance in each of four 
instructional settings between the pretest and posttest: (a) the mean 
number of instructional hours per week a student received; (b) the 
number of hours of instruction each client received for classroom 
instruction only, lab only, independent study only, and class plus lab.^"^ 

7. The total hours of instruction received between pretest and posttest, 
across all instructional environments. 

Principal findings from the regression analyses about direct effects on literacy 
outcomes for adult education clients are summarized below for each program 
component. These findings are supplemented by the ANCOVA results on the 
differential effectiveness of adult education instructional envirorunents. The complete 
statistical results of the regression analyses are presented in Exhibits E.6, E.7, and E.8 
of Appendbc E. The reader should note that all references to CASAS and TABE test 
scores are in terms of scale scores. 

Preview of Findings 

As this chapter will show, clients in each of the three program components 
generally improved their reading ability. Other than client entering ability (as 
measured by the pretest score), there is little overlap across program compon ^nts in 
the predictors of literacy outcomes. Exhibit 3.2 shows, for example, that race/ 
ethnicity was predictive of ESL and ASE literacy outcomes; however, race /ethnicity 
did not influence ABE literacy outcomes. In the area of program characteristics, cost 
per hour of instruction had a direct effect on literacy outcomes for ESL and ASE 



The variables measuring learning environment are derived from item 4 of the Update Record 
with respect to a client's predominant participation in one of the following. 

Lab Only: Computer-assisted or Learning Lab. 

Classroom Only: Class with teacher only or with teacher and aide. 

Independent Study: Self-study or Tutor. 

Class Plus Lab: Class with teacher or with teacher and aide, and computer-assisted or learning 

lab. 
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clients; again, there was no relationship between this factor and literacy outcomes for 
the ABE component. As a third example, overall program attendance as measured 
by total hours of instruction was found to be significantly related to literacy outcomes 
for ESL students only. All of those factors found to be predictive of client literacy 
outcomes, as summarized in exhibit 3.2, are discussed at length in subsequent 
sections of this chapter, particularly those sections which review the principal 
findings for each instructional component. 



Exhibit 3.2 

Factors Influencing Literacy Outcomes for ESL, ABE, and ASE Clients 
Based on Three Multiple Regression Models 



V^ldbb Ul i rtrUlLLUro 


(R Sq := .70) 


(R Sq := .70) 


AQF Modpl 
(R Sq = .49) 


Client 

Characteristics 


Race /Ethnicity 
Education 
Pretest Score 


Pretest Score 
Required Enrollment 


Race/Ethnicity 
Pretest Score 


Program 
Characteristics 


Cost 


Curriculum 
Full-Time Staff 


Cost 

Committed Staff 


Attendance and 

Instructional 

Environment 


Total Hours 
Intensity 


Lab-Only Hours 
Class-Only Hours 





Magnitude of Learning Gains 



Clients who participated in the federal Adult Education Program generally 
increased their reading achievement as indicated by pretest-posttest gains measured 
by the CASAS and TABE. Reading achievement gains were statistically significant 
for each of the three program components (see exhibit E.5 in Appendix E). 

On the basis of the results of the weighted ESL analysis, we estimate that ESL 
clients received a mean of 120 hours of instruction (between the pretest and posttest) 
and attended classes for an average of 14 weeks. Exhibit 3.3 indicates that the 
average ESL client gained 5 scale-score points on the CASAS reading test. As a 
group, ESL clients began instruction with low literacy skills and would generally 
have been capable of holding only entry-level jobs. The CASAS posttest performance 
of ESL clients suggests that, as a group, they had improved their reading skills in 
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fimctionally useful ways important to surviving in society and would now be capable 
of holding jobs, or participating in job training, requiring the comprehension of 
simple text information (Rickard 1988). 

The results of the ABE analysis indicate that ABE students received a mean of 
84 hours of instruction (between pretest and posttest) and attended classes for an 
average of 15 weeks. Exhibit 3.3 indicates that the average ABE client gained 15 
scale-score points on the TABE reading comprehension test. On the basis of, the 
weighted ASE analysis, we estimate that ASE clients received a mean of 63 hours of 
instruction and attended class for an average of 11 weeks. Exhibit 3.3 indicates that 
the average ASE client gained 7 scale-score points on the TABE reading 
comprehension test. The reading ability of the ABE clients at the beginning of 
instruction was equivalent to that of an elementary school student at the beginning of 
the sixth grade (GE = 6.1), whereas the ASE clients generally entered adult education 
with the reading ability of a secondary school student at about the mid-point of the 
eighth-grade school year (GE = 8.5).^^ After instruction, ABE students were reading 
at a level equivalent to that of a student at the end of the first semester of seventh 
grade (GE = 7.4); ASE students were reading at a ninth-grade level following 
instruction (GE = 9.3).^^ 

Exhibit 3.3 

Mean Test Scores and Gains for ESL, ABE, and ASE Clients 



Analysis 
Group 


Test 


N 


Pretest 




Posttest 


Gain 




Mean 


(a) 


Mean 


(a) 


Mean 


(a) 


ESL 


CAS AS 


347 


207 


(15) 


212 


(15) 


5* 


(10) 


ABE 


TABE 


110 


728 


(38) 


743 


(31) 


15* 


(23) 


ASE 


TABE 


154 


755 


(19) 


762 


(17) 


7* 


(14) 



a = Standard deviation 



Statistically Significant (p = .00) 



Grade Equivalent (GE) scores are an often-ixsed reference point for interpreting tests in adult 
education. The GE scale (.0 through 12.9) represents the 13 years of school at the elementary- 
secondary level (K-12) and the 10 months of the traditional school year, using September (.0) and June 
(.9) as the beginning and end points. 

Grade equivalents should be interpreted as stahis measures reflecting performance equivalent to 
students in the norm group (of the California Achievement Test, in the case of the TABE) who have 
completed a particular month of instruction in a graded program. Since GEs are not based on an 
interval scale of measurement, gain score interpretations are inappropriate. For further information, 
see the TABE Norms Book and Table 71, published by CTB-McGraw-Hill, 1990. 
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The Importance of Pretest Reading Performance 

As a preface to discussing the regression results, it should be noted that the 
strongest factor - and the only common factor - influencing literacy outcomes (i.e., 
posttest reading performance) in all three program components was the pretest 
reading achievement score. In education, the pretest is often found to be the strongest 
predictor of posttest performance and this is one reason it is often used as a control 
variable (as it was in this study of learning gains). The pretest score is also 
considered by many educational psychologists to represent student entering ability 
(in the sense of an initial performance level) since it is often found to be statistically 
related to a variety of background influences on achievement that include educational 
attainment, intelligence, and prior knowledge relevant to what is being measured by 
a test. 

The influence of initial reading level (as represented by the pretest) on literacy 
outcomes (as represented by the posttest) was strongest in the case of ABE students 
where approximately 61 percent of the variance in the posttest scores was directly 
attributable to entering ability; that is, over half of the performance of ABE students 
on the posttest can be explained by their performance on the pretest. In the case of 
ESL students, almost half (48 percent) of their learning gains can be accounted for by 
initial achievement levels. Literacy outcomes were least influenced by entering ability 
among ASE clients; here, the pretest accoimted for only 19 percent of posttest 
performance. 

ESL Literacy Effects 

Six factors were identified that directly influence ESL clients' reading 
achievement. In the area of client characteristics, direct effects were found for 
race /ethnicity, prior educational attainment, and entering ability as measured by 
pretest reading score. Cost per client seat hour was the only measured program 
characteristic to be predictive of ESL literacy outcomes. In the area of program 
attendance and instructional environment, both total hours of instruction and hours 
of instruction per week were found to be significantly related to ESL reading 
achievement. 

It should be kept in mind that each of the effects discussed below represents 
average effects on CASAS reading posttest performance after controlling for the 
influence of the 15 factors measured by the 19 variables entered hierarchically in the 
ESL regression model. 

Client Characteristics. Almost half (48 percent) of the variance of the reading 
achievement (posttest scores) of ESL students can be accounted for by their initial 
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level of English reading achievement.^^ On average and everything else being 
equal, white ESL students are estimated to score higher on the CASAS reading 
posttest compared to Hispanic ESL students. ESL students with higher levels of prior 
education are also estimated to achieve higher levels of English literacy on average. 

Program Characteristics. Students attending ESL programs with high costs 
per seat hour (i.e., greater than $4.57 per client per hour of instruction) can be 
expected to perform higher on the CASAS reading posttest compared to ESL students 
attending average-cost programs. ESL students in average-cost programs are also 
expected to achieve a higher level of literacy compared to their peers in low-cost 
programs. 

Program Attendance and Instructional Environment. Only for ESL students 
was total hours of instruction significantiy related to reading achievement. On 
average and everything else being equal, the unique contribution of program 
attendance (as measured by total hours of instruction) to the impact of ESL 
instruction is an increase of 1 scale-score point on the CASAS reading test 
for approximately 40 hours of ESL instruction.^^ The reader should keep in mind 
that there are other factors in addition to program attendance which contribute to the 
five-point average gain for ESL clients. One such factor is intensity of instruction as 
measured by hours of ESL instruction per week. Intensity of instruction, however, 
was fovmd to be negatively related to reading achievement. 

As a follow-up, analysis of covariance was used to determine whether literacy 
outcomes differed significantly across the three principal learning environments (i.e., 
class-only, lab-only, and class-plus-lab), irrespective of the amoimt of instruction in 
those environments. The results of the ANCOVA analyses for ESL (see exhibit E.9 in 
Appendix E) indicated that type of learning environment - considered independently 
of the amount of instruction within an environment - did not produce significantly 
different learning outcomes for ESL students when other significant influences on 
learning were statistically controlled. Having thus established that reading 
achievement does not differ based on the mere exposure of ESL students to a 
particular type of instructional setting, we then established through further regression 
analysis that the effect of total hours of instruction primarily reflects the effect of 
hours of instruction within the dominant ESL learning envirorunent: the classroom- 
only setting. Therefore, it is primarily the amoimt of instruction within an ESL 



The standardized regression coefficient (beta) for the ESL pretest effect is .69; squaring this beta 
value yields .476, or approximately 48 percent. 

^7 The unstandardized regression coefficient for total hours of instruction is .025 (see exhibit E.6 in 
Appendix £). The estimate of 40 hours was obtained by dividing unity (i.e., one) by this value (i.e., 
l/,025 = 40). 
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classroom environment that make a difference in promoting positive literacy 
outcomes for ESL clients. 

In summary, overall program attendance (as measured by total hours of 
instruction) was found to have a positive impact on literacy outcomes for ESL 
students. The impact of overall attendance in the ESL program largely reflects 
student persistence in the dominant ESL instructional envirorunent: participation in 
classroom-only instruction. In addition, program cost per hour of instruction, the 
client's level of prior educational attainment, entering achievement level, and 
race/ethnicity (if white rather than Hispanic) were all positively related to literacy 
outcomes for ESL students. Finally, intensity of instruction (as measured by hours 
per week) was found to be inversely related to ESL literacy outcomes. 

ABE Literacy Effects 

Five factors were identified that directly influence ABE clients' reading 
achievement. In the area of client characteristics, both entering ability (as measured 
by pretest reading score) and enrollment motivation (whether voluntary or required) 
were found to be related to ABE literacy outcomes. Literacy effects related to 
program characteristics were found for curriculum design (if highly individualized) 
and for the presence of full-time staff. In terms of client attendance within particular 
instructional environments, negative effects were found for client persistence in lab- 
only and classroom-only instruction. 

In reviewing each of these considerations below, it should be kept in mind 
that the results pertain to average effects on TABE posttest reading performance after 
controlling for the influence of the 15 factors measured by the 19 variables entered 
hierarchically in the ABE regression model. 

Client Characteristics. Over half (61 percent) of the variance of the posttest 
reading achievement of ABE students can be accounted for by their initial level of 
reading achievement.^^ In terms of enrollment motivation, TABE posttest reading 
performance is estimated to be significantly lower for ABE clients who were required 
to attend adult education compared to those who enrolled volimtarily. ABE clients 
required to enroll in adult education are also about twice as likely to be welfare 
recipients compared to those ABE students who enroll voluntarily. 

Program Characteristics. Students participating in ABE programs offering 
highly individualized curricula are estimated to score significantly higher on the 
TABE posttest compared to ABE students enrolled in programs described by staff as 
offering less individualized or more structured curricula. ABE clients in programs 



The standardized regression coefficient (l^eta) for the ABE pretest effect is .78; squaring this beta 
value yields .608, or approximately 61 percent. 
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with at least one full-time administrator and one full-time instructional staff member 
(full-time staff) are also estimated to score significantly higher on the TABE posttest, 
on average, compared to ABE clients in programs witia no full-time staff. 

Program Attendance and Instructional Environment. An overall effect of 
total hours of instruction on ABE client literacy outcomes was not observed. Neither 
was there an effect of instructional intensity (hours of instruction per week). 
However, the amount of instruction in lab-only and class-only instruction was fotmd 
to be inversely related to reading achievement. That is, compared to a combined 
classroom and lab envirormient, persistence in lab-only and classroom-only 
environments had a negative effect on literacy outcomes for ABE students: posttest 
reading achievement decreased with increasing amoimts of instruction in these two 
learning environments. 

Some practitioners might be tempted to explain the negative effects of 
persistence in the lab-only and classroom-only enviroiunents by appealing to an 
argument which assumes that it must be the lower-ability students who persist 
longer in these two settings. While this is true in the ABE classroom-only 
environment, it must be recognized that the regression analysis controlled for 
entering ability. Thus, the negative effect of persistence in the classroom-only 
environment cannot be explained by lower-ability students staying longer (even 
though they do). In the case of the lab-only envirormient, it was actually the higher- 
ability students who persisted longer (87 hours versus 76 hours, on average). 

The follow-up ANCOVA analyses showed that type of ABE learning 
environment, per se, has no measured effect on adjusted posttest scores. However, 
the ABE regression model indicated that persistence in particular learning 
environments (i.e., the lab-only and classroom-only envirormients) is associated with 
detrimental effects on reading achievement for ABE students. Furthermore, it is 
interesting to note that a negative effect of persistence was not observed for the class- 
plus-lab environment. A theoretical interpretation of these findings would suggest 
that the achievement of positive literacy outcomes for ABE clients may be facilitated 
more by classroom plus lab rather than by classroom-only or lab-only instructional 
settings. 

In summary, overall program attendance (as measured by total hours of 
instruction across type of instructional environment) was not found to have an 
impact on literacy outcomes for ABE students. Furthermore, required attendance and 
persistence in lab-only and class-only instructional environments were foimd to have 
negative impacts on ABE reading achievement. Positive influences on ABE literacy 
outcomes included the clients' entering level of reading achievement and local 
program practices associated with highly individualized curricula and full-time staff. 
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ASE Literacy Effects 

Four factors were identified that directly influence ASE clients' reading 
achievement. In the area of client characteristics, effects were found for 
race/ethnicity and for pretest reading score. With respect to program characteristics, 
cost per client seat hour was found to be positively related to ASE literacy outcomes, 
whereas programs characterized by experienced (i.e., "committed") ASE staff showed 
negative effects on client literacy outcomes compared to programs whose staff had 
less experience in adult education. Length of program attendance and differences in 
instructional envirorunent were not found to have an influence on ASE client literacy 
outcomes. 

Again, the reader should keep in mind that the following discussion is about 
estimates of average effects on TABE posttest reading performance after controlling 
for the influence of the 15 factors in the ASE regression model. 

Client Characteristics. Entering student ability as measured by pretest score 
was foxmd to have a relatively small effect on ASE literacy outcomes, accoxmting for 
approximately 19 percent of tiie variance of ASE client reading achievement, on 
average. Everything else being equal, Asian students participating in ASE are 
estimated to score significantly higher on the TABE reading posttest compared to 
white ASE clients on the average. Alternatively, black non-Hispanic ASE students 
are estimated to score significantly lower on the TABE reading posttest than white 
non-Hispanic ASE clients. 

Program Characteristics. Students attending ASE programs with high costs 
per seat hour (i.e., greater than $4.57 per client per hour of instruction) are estimated 
to score significantly higher on the TABE reading posttest compared to ASE students 
attending average cost programs. ASE students in average-cost programs are also 
expected to achieve a higher level of literacy compared to their peers in low-cost 
programs. Perhaps surprisingly, ASE students in programs where a majority of 
teachers are certified in adult education or have at least three years of adult 
education teaching experience scored significantly lower, on the TABE reading 
posttest, on average, compared to ASE students in adult education programs not so 
characterized. The expectation that programs with experienced or certified staff 
should produce better literacy outcomes compared to programs not so characterized 
was not borne out by the data. 

Program Attendance and Instructional Environment. As with the ABE 
component, an overall effect of attendance (total hours of instruction) was not 
obsei-ved for ASE students with respect to impacts on client literacy. Neither was 
there observed any effect related to intensity of instruction or to participation in 
different types of instructional environments. 
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In summary, positive impacts on ASE literacy outcomes were found for (1) 
client race/ethnicity (if Asian compared to white, or if white compared to black), (2) 
clients' level of initial reading achievement, and (3) program cost per student per 
hour of instruction. ASE programs in which a majority of the teachers are certified in 
adult education or have at least three years of teaching experience in adult education 
(i.e., "committed" programs) were not found to be associated with better literacy 
outcomes. 

Summary and Conclusions 

Clients in each of the three instructional components improved their reading 
achievement during the time they participated in adult education. ESL clients gained 
an average of 5 scale-score points on the CASAS reading test while ABE and ASE 
clients gained an average of 15 and 7 scale-score points respectively on the TABE 
reading comprehension subtest. 

With few exceptions, client literacy outcomes were foimd to be influenced 
largely by different factors in each of the three adult education components. One 
exception was the influence of entering student ability: pretest score was the 
strongest predictor of reading achievement in all three instructional components. The 
closest the data come to offering a common prescription for effective program 
practices is in the area of cost. In both the ESL and ASE components, higher cost per 
student-hour of instruction was foimd to be positively related to client learning gains. 

Total hours of instruction was observed to be related to client literacy 
outcomes only in the ESL component. In both the ABE and ASE programs, total 
hours of instruction was not significantly related to client literacy outcomes. In short, 
instructional effects associated with client persistence were observed for ESL students 
but not for ABE and ASE students. 

A common assumption among education policy makers is that if the amount 
of instruction provided increases, there will be a corresponding increase in "on-task" 
time by students and that student achievement will therefore increase. The results of 
our analyses with respect to the relationship between persistence and instructional 
effects are not clear-cut, however. We suspect that the relationship between hours of 
instruction received and measured instructional effects may involve the interaction of 
program characteristics with instructional time in ways that this study did not fully 
measure. 

The need to conceptualize program effectiveness in ways that are context- 
specific is evident from the fact that program-related instructional effects were 
generally very specific to each of the three adult education components. In the ESL 
component, cost and total hours of instruction were the significant program factors 
that positively influenced client literacy outcomes, and the program factor measuring 

42 



O ) 



intensity of instruction (i.e., hours of instruction per week) was negatively related to 
ESL literacy outcomes. For ABE students, program factors positively related to 
reading achievement included curriculum designs which were highly individualized 
and programs with at least some full-time staff. It would also appear that student 
persistence in ABE instructional environments which combine classroom and 
laboratory instruction may be optimal for facilitating client literacy outcomes. The 
only program variable found to be predictive of literacy outcomes in more than one 
instructional component was cost per hour of instruction. The cost factor was 
observed to have positive effects on client literacy outcomes in both the ESL and ASE 
components. 

Recommendations 

In closing this chapter, it is important to comment on the limitations of the 
learning gains study. First, the learning gains analysis was exploratory and design 
tradeoffs had to be made in data collection in order to maximize program 
cooperation and participation with respect to other objectives of the national 
evaluation (e.g-, to describe the federal Adult Education Program and patterns of 
attendance and persistence as opposed to emphasizing assessment and instructional 
issues). Second, the final analysis samples were relatively small, which in large part 
reflects the quantity and quality of the test data reported by local programs. This is 
not to say that local program staff are generally incompetent in the area of client 
assessment. Rather, our experience should be viewed as signaling that there are 
problems with assessment practices in adult education which have to do with local 
program capability, local testing procedures, as well as the quality of the 
standardized assessment instruments currently available. 

We suspect that there are four fundamental problems in adult education 
assessment which need to be better understood if similar evaluations are to be 
attempted in the future: 

• Need for Technical Assistance: The Adult Education Program, because 
of the part-time nature of its staffing, staff turn-over, and lack of staff 
training in assessment, is unlikely to produce good test data without 
technical assistance. We suspect that the problem of floor and ceiling 
effects could have been reduced had functional-level testing been 
practiced by local programs. Yet even with technical assistance, the 
capability for generating adequate assessment data remains questionable 
because most local programs simply do not have the staff resources and 
person-hours to devote to assessment. 

• Obstacles to Testing: Clients often do not like to be tested, and in some 
cases, because of the open-entry /open-exit nature of client attendance, 
they cannot be. Also, adult education clients complete their programs 
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of study relatively quickly which often results in lost assessment 
opportunities. 



• Dissatisfaction with Available Instruments: Adult education staff tend 
to dislike the assessment process and the available instruments. This is 
understandable in view of the need for greater measurement sensitivity 
in tests like the TABE, where ceiling effects are apparently a major 
problem. The measurement accuracy problem in adult education is 
compounded by the wide range of individual performance differences 
which appear to be much greater than that generally measured by 
current instruments. The pervasiveness of ceiling effects encoimtered in 
the learning gains study suggests that test developers could improve the 
measurement accuracy of current adult education assessment 
instruments by expanding test levels designed to measure higher levels 
of adult achievement.^^ 

• Lack of Consensus in the Field: Testing practices vary widely from 
state to state and from one locality to another. The consequence is that 
there are few, if any, standards for guiding appropriate assessment 
practices in adult education. In California, for example, professional 
opinions are extremely divergent about appropriate ESL baseline 
assessment practices. 

Until the above assessment issues are resolved, evaluation in adult education 
will remain problematic. What is clearly needed in future evaluations of adult 
education is greater research control over the outcome data, regarding not only the 
choice of research design but also to the provision of local training in assessment 
procedures that can be monitored. Our impression is that most adult education 
programs need technical assistance in assessment and would be receptive to such 
help. Finally, in the interest of producing high-quality program research that meets 
the need for both internal and extemal validity, it might be best to focus future adult 
education impact evaluations on small-scale demonstrations. 



In the learning gains study, ASE students scoring within the ceiling range of the TABE had a 
significantly higher level of prior educational attainment compared to ASE students measured more 
accurately by the TABE. This was not the case for ABE students. This finding suggests that the TABE 
could be made more sensitive in measuring higher levels of adult achievement by adding a fifth level 
of item difficulty geared toward higher-ability ASE students. It may also reflect the possibility that a 
substantial number of ASE clients enter the program already possessing the requisite skills for a GED 
(28 percent of the 2,250 ASE clients with pretests scored at the 12.9 grade level on the TABE), and that 
for them the program serves more as a credentialing than as an educational fimction. 
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Chapter 4 

EDUCATIONAL ATTAINMENT AND 
EMPLOYMENT-RELATED OUTCOMES 



Adults enroll in instructional programs to achieve any number of 
objectives,^^ including the three specified by the Adult Education Act applicable to 
federally supported local programs: (1) to enable clients to acquire basic education 
skills necessary for literate fimctioning; (2) to benefit from job training and retraining 
programs so that they can obtain and retain productive employment; and (3) to offer 
clients the opportimity to continue their education to at least the level of completion 
of secondary school. 

> 

The extent to which clients have acquired basic education skills can be 
assessed in a number of ways. In chapter 3, we presented findings on client gains in 
reading achievement as derived from standardized test data. Two additional types of 
evidence related to the acquisition of basic education skills are presented in this 
chapter. These include findings about client advancement in instructional placement 
and findings about client perceptions of the extent to which the program helped them 
improve basic education skills. In general, beginning students advanced the most in 
placement levels and over half of all participants (54 percent) believed that the 
program had helped them "a lot'* in acquiring one or more basic skills. 

Employment-related benefits of program participation are also reported in this 
chapter. Specifically, we present findings on net employment gains and client 
perceptions of employment-related benefits attributable to the program for those who 
were employed and unemployed when they enrolled. We estimate a net gain of 
about 6 percentage points in employment for program participants. Typically, what 
participants learned in the program helped them to retain employment or to improve 
their performance in the job they had prior to enrollment rather than helping 
unemployed participants to find a job. 

We also report estimates regarding the extent to which clients completed 
secondary education and continue on to higher education. Depending on the criteria 
used, we conservatively estimate that adult education enabled between 11 and 30 
percent of the ASE participants to complete their secondary education (or, 4 to 12 
percent of all participants). We also estimate that 17 percent of the former clients are 
currently continuing their education and that at least 7 percent of the program's 



For a discussion of client motivations for enrolling in adult education programs, see pages 52-55 
and Appendix D in Young, et al. (September, 1993). Second Interim Report o f the National Evaluation 
of Adult Education Programs: Profiles of Client Characteristics. Arlington, Virginia: Development 
Associates. 



45 



participants were enrolled in post-secondary education 6 months after leaving the 
federal Adult Education Program. 

In addition to providing estimates on the extent to which clients benefitted 
with respect to the three objectives specified by the Adult Education Act, this chapter 
presents findings on enhancement of client self-image, increases in the frequency of 
clients' reading to young children in their household, and clients' reasons for leaving 
adult education. As reported later in this chapter, we found that approximately 70 
percent of the clients accomplished personal goals related to the enhancement of self- 
image. We also found a net increase of 15 percent in the number of clients who read 
more often to their young children. For ABE and ESL clients, the primary reasons for 
leaving adult education are related to events external to the program, usually 
pertaining to changes in employment; ASE students most often leave adult education 
because iJiey completed their program of study. 

The Telephone Survey 

Most of the data and analyses in the chapter are based on the telephone 
follow-up survey. Before reviewing the education and employment-related findings, 
several characteristics of that survey are important to note, fhe follow-up survey 
was completed by telephone with a sample of 5,401 clients^^ from 109 local 
programs; these respondents had completed a Client Intake Record Form B and were 
known to have been out of the program for 6 months. All 5,401 of the clients in the 
telephone follow-up survey were asked to confirm their participation in the adult 
education program, to indicate the number of classes they attended, and to tell us the 
major reason(s) they left their class or instructional program. The bulk of the 
telephone survey, however, reflects the views of 4,653 respondents (86 percent of 
those contacted) who attended at least three adult education classes. These survey 
items focused on client perceptions of program benefits related to basic skills, 
employment or other areas; plans for further education; and the nature and quality of 
the instruction they received. The subsample of clients who attended at least three 
classes represents, when weighted, a national population of 736,259 clients, or about 
half of active new clients. As described in Appendix C, the weighting of the 
telephone survey essentially adjusts the results for non-response bias. After the 
weight adjustments were applied to the telephone survey, respondents are very 
similar to the clients in the national sample who began instruction,^ except for 



Telephone interviews were attempted with a sampling jframe of 10,500 clients. The 5,401 
completed interviews reflect a 51 percent response rate. 

^ The weighted total telephone survey sample (N = 5,401) reflects an estimated national 
population of 86?.,508 new clients who enrolled in adult education between April 1991 and April 1992, 
and who had been out of the program for at least 6 months by April 1993. Eighty-six percent of the 
sample attended adult education for at least three classes; 11 percent attended for 1-2 classes; and 3 
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having received fewer Hours of instruction and having been out of adult education 
for six months. 

Benefits in Basic Educational Skills 

As discussed in Chapter 3, new clients in each of the three instructional 
components improved their reading achievement during the time they participated in 
adult education. ESL clients gained an average of 5 scale-score points on the CASAS 
reading test. ABE and ASE clients gained an average of 15 and 7 scale-score points 
respectively, on the TABE reading comprehension subtest. 

Changes in Placement Level. ESL and ABE clients also made gains with 
respect to changes in placement levels. (Because ASE is the highest instructional 
level, clients who entered in this component are excluded from this analysis.) In 
general, placement changes reflected advances in instructional placement and these 
were greater for beginning students compared to intermediate or advanced placement 
students. As noted in exhibit 4.1 below, 41 percent of the beginning ESL students 
advanced in placement level over the course of the study (most of whom advanced to 
ESL intermediate) and 30 percent of the beginning ABE students advanced in 
placement (most of whom advanced to ABE intermediate). This compares to 
advances in placement for 28 percent of the intermediate ESL group, 12 percent of the 
advanced ESL group (who went on to ABE or ASE), and 21 percent of the 
intermediate ABE group (who went on to ASE).^ 

EXHIBIT 4.1 
Changes in Instructional Placement 




Client Perceptions of Skill Improvement. Most clients indicated that 
improvement in literacy skills was a basic motivation for enrolling in adult education. 
For example, 79 percent rated improvement in reading/writing skills as very 
important while 69 percent rated improvement in speaking/ listening skills as very 
important. In the follow-up survey, respondents were asked to rate the degree to 



percent dropped out of the program after enrollment without attending any classes. 
See exhibit 5.1 (chapter 5) for more detail. 
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which adidt education classes or training programs had helped (a lot, somewhat, or 
not at all) to improve their skills in those areas which they had rated as very 
important at enrollment. These results are displayed in exhibit 4.2 for clients who 
indicated that adult education had helped them improve "a lot" in three specific areas 
of basic skills which they had rated as very important at intake. 

Exhibit 4,2 

Proportion of Clients Saying That Adult Education 
Had Helped Them "a Lot" in Basic Skills Rated as Very Important 

(N = 3,803) 



Basic Skill Areas 


Percent of Clients 


ESL 


ABE 


ASE 


Overall 


Reading and writing 


44 % 


50% 


45 % 


46 % 


Mathematics 


26 


51 


49 


42 


Speaking and listening 


48 


48 


45 


47 


Total helped in at least one skill area 


62 


68 


63 


64 



Note: Exhibit 4.2 reflects follow-up responses of clients who had indicated at 
enrollment that a particular skill area was very important to them; percentages in 
exhibit 4.2 do not reflect the ratings of all clients in the follow-up sample. 



The percentages in the bottom row of exhibit 4.2 indicate that a majority of 
clients believe they were helped '*a lot" in at least one of the three basic skills areas 
rated as very important to them; overall, 64 percent of these clients had been helped 
"a lot" by adult education instruction. Since improvement of basic math skills was of 
primary importance to ABE and ASE students but not to ESL students, it is not 
surprising that only 26 percent of the ESL clients indicated great improvement in the 
area of math skills. Considering the responses of all clients, irrespective of their 
importance ratings for improvement in basic skills, the percentage helped in at least 
one basic skill area is: 60 percent ESL, 61 percent ABE, 53 percent ASE, and 58 
percent overall (see exhibit 4.22 at the end of the chapter). 

Generally, client opinion about improvement in basic reading skills was 
consistent with test score gains. As shown in exhibit 4.3, client opinion and test score 
gains in the area of reading improvement converged^^ for the majority of ABE, ASE, 



Convergence was defined as a match between client opinion of whether the program helped "a 
lot" in improving their reading or writing skills and a reading test score gain. A match was also 
scored ii the client said the program helped them "somewhat" or "not at all" and their test score gain 
was zero or negative. Divergence was defined as a lack of a match between client opinion and test 
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and ESL students for whom both test score and follow-up interview data were 
available. Although the ntimber o ' cases available for comparison is small, we are 
able to reject the hypothesis that the results, across the three instructional groups, are 
due to chance. 

Exhibit 43 

Consistency Between Client Opinion and Test Score Data 
Regarding Improvement in Basic Reading Skills 

(N = 405) 



Consistency 


ABE 


ASE 


ESL 


Total 


Category 


(126) 


(196) 


(83) 


(405) 


Divergence 


40% 


45% 


40% 


42% 


Convergence 


60% 


55% 


60% 


58% 



Below, exhibit 4.4 compares the amount of instruction received by clients who 
reported being helped '*a lot" (i.e., benefitted) in at least one area of basic skills with 
clients who did not indicate such benefits of instruction. As the exhibit shows, clients 
who reported having benefitted from adult education received considerably more 
hours of instruction than average. This appears to be particularly true for ABE 
clients. 



Exhibit 4,4 

Relationship of Amount of Instruction to Reported Benefits in Basic Skills* 

(JV= 3,803) 



Program 
Component 


Median Hoiirs of Instruction 


Median Weeks of Enrollment 


Benefitted 


No Benefit 


Benefitted 


No Benefit 


ESL 


105 


88 


18 


12 


ABE 


46 


24 


12 


7 


ASE 


35 


28 


10 


8 


Overall 


48 


30 


10 


8 



* Clients who benefitted reported the program had helped them "a lot" in one or more skill 
area: reading and writing, math, or speaking and listening. 



score change (e.g., positive opinion and negative or zero test gain). 
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Employment-Related Benefits 

The desire to improve one's employability is a major motivation of many new 
clients who enroll in adult education. As we have reported elsewhere,^ new clients 
motivated by employability goals are often interested in benefitting from adult 
education in one of two general ways. Those who are unemployed typically aspire to 
becoming employed and hope that adult education can help them achieve such 
employment. Alternatively, improving current job performance through the 
acquisition of new skills — whether these be vocational skills, basic academic skills, or 
language skills - is often a desire of those who are already employed when they 
enter adult education. Somewhat surprisingly, the highest degree of motivation for 
the enhancement of employability skills was among ESL clients. And consistent with 
that earlier finding, it is clear from the data reported in this chapter that ESL clients 
are also the ones who most often experience the employment-related benefits of 
participating in adult education. 

At the time of enrollment in adult education, the labor force status of the 
follow-up sample (excluding those who attended fewer than three classes) was 
distributed as shown in exhibit 4.5 below. In general, a plurality of the new clients 
were employed when they began instruction. ESL clients were more likely to be 
employed at the time of enrollment than were ABE or ASE clients. Percentages in 
the Total column of exhibit 4.5 do not sum to 100 because of roimding. 



Exhibit 4.5 

Labor Force Status of .the Follow-up Sample 
At Enrollment in Adult Education 



Status at Intake 


ABE 


ASE 


ESL 


Total 


Employed 


41% 


42% 


48% 


44% 


Unemployed 


26% 


29% 


19% 


25% 


Not in Labor Force 


33% 


29% 


33% 


32% 


Sample Size 


N = 1096 


N = 1696 


N = 1596 


N = 4388 



Our strategy for the balance of this section is to describe the change in 
employment status between intake and follow-up by tracking those clients in the 
telephone sample who were in the labor force at the time of enrollment. Our 
principal interest is not only to examine changes in emplojonent status (i.e., being 
either employed or unemployed) for those clients who entered adult education as 
members of the labor force but also to focus on what the clients claim to have gotten 



^ See Young, et al. (September 1993). Second Interim Report of the National Evaluation of Adult 
Education Programs: Profiles of Client Characteristics. Arlington, Virginia: Development Associates. 
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from participation in adult education as it affected their employability under four 
possible employment-change conditions: (a) ceasing to be employed; (b) remaining 
employed; (c) still not employed; and (d) changing from unemployed to employed. 

Exhibit 4.6 provides a summary of the before- and after-employment status of 
the follow-up sample. Cell A indicates that 13 percent of all clients were employed 
at program intake, but were not working 6 months after they left adult education. 
Cell B shows that 50 percent of all clients were employed at intake and were also 
employed six months after leaving the program. Cell C indicates that 18 percent of 
all clients were unemployed at intake and were also unemployed six months after 
leaving the program. Finally, Cell D shows that 19 percent of all clients were 
unemployed at intake, but were employed six months after leaving the program. 



Exhibit 4.6 

Before-After Changes in Employment Status 
For Clients in the Labor Force at Intake 
(N = 3,042) 



After 



Before 

Employed 
Unemployed 



Not 
Employed 



Employed 



(A) 


13 % 


(B) 


50 


% 


63 % 


(C) 


18% 


(D) 


19 


% 


37% 



31 % 



69% 



The data in the exhibit shows a net gain of six percentage points in 
employment rate, from 63 percent at intake to 69 percent at the time of follow-up, or 
six months after leaving the program. The difference is statistically significant. This 
is not to say th:it the increase in employment was caused by participation in adult 
education, merely that this was the amount of change in emplojnnent observed. 
However, we do examine in the following discussion the extent to which clients in 
each of these four cells attributed employment-related benefits to participation in 
adult education. 

Clients Who Became Employed (Cell D). Clients who were imemployed at 
enrollment but who became employed 6 months after leaving adult education (19 
percent of the clients in the labor force at intake) were asked in the follow-up 
interview if what they had learned in the program had helped them get their job. 
Based on the statistically significant responses (see exhibit 4.7 below), the overall 
answer is "no" since 57 percent (versus 43 percent) of these respondents indicated 
that program participation had not been instrumental in their having obtained 
employment. The overall perspective is primarily that of the ABE and ASE students. 
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However, the reader should note that two-thirds of the ESL respondents did, in fact, 
claim that what they had learned in adult education had made a difference in their 
becoming employed. 

Exhibit 4.7 

Changing From Unemployed to Employed: 
"Did What You Learn in the Program Help You Get the Job?" 



Client Response 


ABE 


ASE 


ESL 


Total 


Yes 


36% 


38% 


67% 


43% 


No 


64% 


62% 


33% 


57% 


Sample Size 


N = 111 


N = 221 


N = 87 


N = 419 



Clients Who Remained Employed (Cell B). Clients who were employed at 
intake and who had remained employed after leaving adult education (50 percent of 
those in the labor force at intake) were asked a number of questions about whether 
program participation had been beneficial to their employment situation. For 
example, clients who still held the same job as when they enrolled were asked if 
what they had learned in the program had been helpful to them in that job. The data 
on this issue are presented below in exhibit 4.8 and indicate that, again, it is primarily 
the ESL clients who benefitted from adult education with respect to their enhanced 
employability. Program participation was as likely as not to benefit ABE students in 
their employment situation, and a majority of ASE students indicated that program 
participation had not helped them in their job. 

Exhibit 4.8 
Remaining Employed: 
"Did What You Learn in the Program Help You with That Job?" 



Client Response 


ABE 


ASE 


ESL 


Total 


Yes 


48% 


40% 


88% 


61% 


No 


52% 


60% 


12% 


39% 


Sample Size 


N = 233 


N = 360 


N = 392 


N = 985 



A number of clients were also working a second job in addition to the one 
they held when entering the program. These clients were asked in the follow-up 
interview if what they had learned in the program had helped them get that second 
job. These data are displayed in exhibit 4.9, the results indicating that it was the ESL 
group that was primarily helped by adult education in attaining additional 
employment. 
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Exhibit 4.9 
Remaining Employed: 
"Did What You Learn in the Program Help You Get That Second Job?" 



Client Response 


ABE 




ESL 


Total 


Yes 


25% 


30% 


70% 


41% 


No 


75% 


70% 


30% 


59% 


Sample Size 


N = 35 


N = 61 


N = 45 


N = 141 



Other clients who remained employed changed to another job after leaving 
adult education. These clients were asked if what they had leamed in adult 
education had helped them to get a better job than the one they ?iad before enrolling 
in the program. The data in exhibit 4.10 once again indicate that it is primarily the 
ESL clients who had been helped to obtain a better job. Most of the ASE students 
who reported changing jobs indicated that the program had not helped them get a 
better job. The program was as likely as not to have helped ABE students to 
improve their job situation. 

Exhibit 4.10 
Remaining Employed: 
"Did What You Leam in the Program Help You Get a Better Job?" 



Client Response 


ABE 


ASE 


ESL 


Total 


Yes 


53% 


44% 


80% 


59% 


No 


47% 


56% 


20% 


41% 


Sample Size 


N = 106 


N = 206 


N = 179 


N = 491 



Clients Who Were Still Not Working (Cell C). Eighteen percent of those in 
the labor force at intake who were unemployed remained out of work at follow-up. 
Some of these clients had, however, worked at some time during the 6-month period 
following their program participation. These clients were asked in the follow-up 
interview whether what they had leamed in the program had helped them get a job. 
For most of the clients in this small sample, the answer was "no." Exhibit 4.11 
displays the results. 
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Exhibit 4.11 
Remaining Unemployed: 
••Did What You Learn in the Program Help You Get a Job?" 



Client Response 


ABE 


ASE 


ESL 


Total 


Yes 


48% 


20% 


0% 


28% 


No 


52% 


80% 


100% 


72% 


Sample Size 


N = 26 


N = 34 


N = 8 


N = 68 



Clients Who Ceased to Be Employed (Cell A). Thirteen percent of those in 
the labor force at intake who were employed were not working after le' ving the 
program. Some of these clients had worked at some time after leaving the program. 
These clients were asked if what they had learned in the program- had helped them 
with respect to their post-program employment situation. Overall, this group was 
about as likely as not to have benefitted from adult education with respect to their 
employability after leaving the program. That is, the proportional distribution in the 
dichotomous response option observed for the total sample in exhibit 4.12 (i.e., 56 
percent Yes versus 44 percent No) is not statistically different from what would be 
expected by chance (which is 50:50), given the sample size (N = 219). This is also 
true of the ABE group. Whereas most of the ASE group who had become 
imemployed indicated that what they had leamed in the program had not helped 
their employment situation, thr. was not the case for ESL clients. Although they had 
become unemployed after the program, the vast majority of these ESL clients claimed 
that participation in adult education had enhanced their employability. 

Exhibit 4.12 
Becoming Unemployed: 
"Did What You Learn in the Program Help You with a Job?" 



Client Response 


ABE 


ASE 


ESL 


Total 


Yes 


45% 


36% 


82% 


56% 


No 


55% 


64% 


18% 


44% 


Sample Size 


N = 58 


N = 78 


N = 83 


= 219 



In concluding the discussion of employment-related benefits, we examine the 
extent to which employability was enhanced for aggregates of clients who were 
employed at the follow-up point and who indicated that the program had helped 
their employment situation. That is, considering those clients who gave affirmative 
responses to those issues covered by exhibits 4.7 through 4.10, che question being 
asked is "How many clients attributed an improvement in their employment status to 
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what they learned in adult education?" The pie charts in exhibit 4.13 answer this 
questioM from the standpoint of the clients' labor force status at the time of 
enrollment. 



Exhibit 4.13 

Proportion of Qients Reporting That What They Learned Helped 
Their Employment Situation and the Median Hours of Instruction They 

Received 



55 hoxirsof 
instruction 

90% of clients 



85% of clients 
37 hoitis of 
instruction 



1C 



G 



54 hours of 
instruction 
10% of clients 



Qients Not In Labor Force 
at Tune of Intake (32%) 



16 %of clients 
39 hours of 
ii^truction 
Qients Unemplojred 
at Time of Intake 0^%) 



50%of dients 
42 hours of 
instruction 



3 



50% of dients 

32 hcnirs of 
instruction 



Qients Employed 
at Tune of Intake (32%) 



71% of clients 



42 hours of 
instruction 




All Qients 



29% of clients 
43 hours of 
instruc±ion 



Program 
helped 



Program didn't 
help 



The trend in the pie chart data is that employment-related benefits are directly 
proportional to one's initial orientation to the labor market. That is, employability 
was most often enhanced for those who were employed when they enrolled in adult 
education. At the other extreme, employability was enhanced the least for those not 
in the labor force at intake. Altogether, approximately 29 percent of the cUents in the 
follow-up sample claimed that participation in aduh education had improved their 
employment situation (see also exhibit 4.22 at the end of the chapter). 
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Continuing Education and Secondary School Completion 

Another legislative purpose of the Adult Education Act is to "enable adults 
who so desire to continue their education to at least the level of completion of 
secondary school." Depending on the criteria used, an estimated 11 to 30 percent of 
the ASE participants had completed their secondary school education within 6 
months after leaving the program. Of ASE clients who began instruction without a 
high school diploma, 11 percent were enrolled in post-secondary education and an 
additional 19 percent had p\ax\s to do so within the next year (this constitutes 28 
percent of all ASE clients or 12 percent of clients overall). 

Continuing Education. To assess the extent to which the continuing education 
purpose of the Act was being accomplished, we asked former clients whether they 
were "attending any educational class or training program now," and if so, what 
kind. As can be inferred from the far right column in exhibit 4.14, approximately 17 
percent of the clients were continuing their education 6 months after having left the 
federal Adult Education Program: about 5 percent were in a high school level 
program, 7 percent in postsecondary, and 5 percent in English language skills. ESL 
students were primarily pursuing further English language instruction; about half of 
the ABE students were enrolled in GED preparation courses; and most of the former 
ASE students who were continuing their education were eruroUed in community or 
regular college classes or in vocational or job training programs. 

We also asked whether the former clients planned to enroll in any educational 
or training classes in the future. Those who said they had such plans (72 percent) 
were asked when and what kind of class or training program they expected to take. 
Exhibit 4.15 indicates that almost half of the former clients (46 percent) intended to 
continue their education in some fashion during the present year whereas 26 percent 
planned to pursue further education the following year. 
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Exhibit 4.14 

Current Continuing Educational Status of Clients Without 
A High School Diploma or the Equivalent When They Enrolled 

(N = 3,114) 



Status of Clients 6 Months After Leaving 
Adult Education 


Percent of Clients Without a High School 
Diploma 


ESL 


ABE 


ASE 


Overall 


Enrolled in GED/high school program 


3 % 


7 % 


6% 


5 % 


Enrolled in conuntmity college or 
vocational training 


4 


5 


11 


7 


Enrolled in Engliish-language skills class 


17 


2 


1 


5 


Not currently attending any educational 
classes or training program 


76 


86 


82 


83 



Exhibit 4.15 

Continuing Education Plans of Clients by Instructional Component 

(N = 4,279) 



Type of Class or Program in Which 
Clients Plan to Enroll "Before Next Year" 


Percentage of All Clients 


ESL 


ABE 


ASE 


Overall 


GED/high school program 


6 % 


22 % 


17 % 


14 % 


Community college/ vocational training 


17 


17 


25 


20 


English language skills class 


22 


4 


3 


10 


Some other class 


1 


3 


2 


2 


Total of all with plans for next year 


46 


46 


47 


46 


None, but plan to enroll "next year" 


31 


23 


24 


26 


None, either had no plans or plans are 
more than a year away 


23 


31 


29 


27 



Completion of Secondary Educ^'ion. The study did not include a direct 
measure of whether the program helpec clients obtain the equivalent of a secondary 
school diploma or to continue their pursuit of that goal. It does, however, provide the 
basis for three relatively conservative estimates of the extent to which clients 
accomplished this purpose of the Act. 

The first estimate is based on former ASE clients who did not have a high 
school diploma when they enrolled and who indicated in the follow-up survey that 
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they were oirrently enrolled in post-secondary education. This represents 11 percent 
of ASE clients (see exhibit 4.16). As the exhibit also shows, some ABE and ESL 
clients who entered without a diploma also indicated that they were enrolled in post- 
secondary level training, and an additional 6 percent of ASE clients indicated they 
were currently enrolled in a high school/GED preparation program, and thus were 
continuing their education in a manner consistent with the purposes of the Act. 

Exhibit 4.16 

Estimates of All Clients Who Were Helped to Continue Their Education at Least 
To the Level of Secondary School Completion 
(N = 437) 





Percent of Clients 


Client Characteristics 


ESL 


ABE 


ASE 


Overall 


Had no diploma, currently enrolled 
in high school/GED 


3 % 


7% 


6% 


5 % 


Had no diploma, currently enrolled 
in community college /vocational 
trcuning 


4 


5 


11 


7 


Total enabled to continue 


7% 


12% 


17% 


12% 



The second estimate is based on former ASE clients who did not have a high 
school diploma when they enrolled and who indicated on the follow-up survey that 
they were currently enrolled in post-secondary education or had plans to do so 
within the coming year. This represents 30 percent of ASE clients. As exhibit 4.17 
shows, 11 percent reported that they were currently enrolled in post-secondary 
courses and 19 percent indicated that they had plans to enroll in such courses during 
the coming year. It should also be noted that an additional 6 percent were currently 
enrolled in a high school/GED program and another 16 percent indicated that they 
had plans to enroll in a secondary school level course at ^ometime during the next 12 
months. 

Tlie third estimate is based on former ASE clients who reported that they had 
completed their adult education program and who are projected to have passed the 
GED test. In the telephone follow-up survey, some 39 percent of the ASE clients 
reported that they left adult education because they had completed the program. 
Assuming these clients took the GED test and that their success rate was the same as 
the average of all those taking the test (70 percent),^^ approximately 27 percent of 



According to the GED Testing Service, about 71 percent of all those who completed the full GED 
test battery during 1993 successfully passed the test. 
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the ASE participants may be assumed to have passed the GED, thereby signifying 
completion of secondary education. The estimate of projected GED passers is 
approximately 10 percent of the total number of clients served by the federal Adult 
Education Program. 

EXHIBIT 4.17 

Continuing Education of ASE Clients Without a High School 
Degree When They Enrolled 
(N=l,706) 

ASE Clients 
Without a Diploma 



Currently 
Enrolled 
18% 



No Plans 
to Enroll 
44% 



PlantoEiuoll 
V^Athin 12 months 
38% 



Postsecondary GED/H5 Other Postsecondary GED/HS 

11% 6% 1% 

Note: Percents are of ABE/ ASE dients who did not have a high school degree or equivalency when they enroUed in the program. 



Other 
3% 



Accomplishment of Other Client Goals 

Adult learners typically entered adult education with multiple motives. In 
addition to goals related to improving basic skills and enhancing employability, many 
clients have personal goals related to improving their self-image. For parents in 
particular, encouraging their children's reading ability is a related aspiration. 

Enhancing Self-Image. In the telephone follow-up survey, we asked clients 
who had indicated that they sought to "feel better about myself/' "contribute better to 
family and community/' "help my children with schooh/ork/' "become less dependent 
on others for help/' or "make others feel better about me" the extent to which the 
program had helped them reach these goals ("a lo^", "somewhat" or "not at all"). A 
summary of their responses is presented in exhibit 4.18. 
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Exhibit 4.18 

Proportion of Clients Reporting That the Program Had Helped "a Lot" 
As Related to Different Reasons for Enrolling in Adult Education 

(N = 4,653) 





Percent of Clients 


Rpa<^on far RnrolliTiP' 


ESL 


ABE 


ASE 


Total 


Feel better about myself 


62 % 


68 % 


66 % 


65 % 


Make others feel better about me 


49 


50 


50 


50 


Contribute to my family and 
community 


50 


46 


40 


46 


Be less dependent on others for help 


47 


44 


44 


45 


Help my children with their 
schoolwork 


31 


38 


33 


33 


Total helped in one area or more 


70 % 


70 % 


68 % 


70 % 



On the whole, most clients reported that the program helped "a lot" with 
achieving one or more of these self-image goals. Overall, 65 percent of those wanting 
to feel better about themselves believed that the program helped "a lot." Some 50 
percent of those wanting others to feel better about them reported that the program 
had helped "a lot" toward achieving this goal, as did 46 percent of those wanting to 
contribute to family and community, and 45 percent of those wanting to be less 
dependent on others. Thirty-three percent of those wanting to help their children 
with schoolwork believed that the program had helped "a lot" toward attaining this 
goal. Overall, 70 percent of the clients reported that they had been helped "a lot" in 
achieving personal goals related to the erihancement of self-concept. 

Reading to Young Children. A program outcome of interest to many adult 
educators is the extent to which participation in adult education programs spills over 
to benefit other members of a participant's household. Particularly interesting in this 
regard, and an explicit purpose of the National Literacy Act (Part B/Title El, Even 
Start Family Literacy Programs), is the effect on yoting children in the household of 
the adult education student. One of the best predictors of a child's academic success 
is being read to and having access to books in the home. 

To assess whether the program resulted in an increase in the frequency of 
reading to children, clients were asked at intake if there were children imder the age 
of 6 in the home and, if so, how often the client read to or with them ("almost never," 
"about once or twice a week," "about once a week," or "nearly every day"). This 
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question was asked again in the telephone follow-up. The results are presented in 
exhibit 4.19. 



Exhibit 4.19 

Proportion of Clients Reporting Changes in How Often They Read to or with 
Their Children Between Intake and Telephone Follow-up 

iN= 1,290) 



Change in Reading Frequency 
between Intake and Follow-up 


Percent of Clients 


ABE 


ASE 


ESL 


Overall 


More often 


38 % 


32 % 


27% 


32 % 


The same 


50 


56 


46 


51 


Less often 


12 


12 


27 


17 


Net increase 


26 % 


20 % 


0 % 


15 % 



Overall, 32 percent reported reading to, or with, young children more often at 
follow-up than at intake while 17 percent reported reading to, or with, these children 
less often at follow-up than at intake, for a net increase of 15 percent in the number 
of clients reading more frequently with young children. The net increase of 15 
percent reflects changes in the ABE and ASE populations but not, surprisingly, 
overall changes in the ESL clients. ABE and ASE clients increased their reading 
involvement with young children on the average from weekly to daily. The overall 
lack of change in tiie ESL group reflects an average pattern of continued weekly 
involvement in reading with young children. The change among the ABE clients is 
particularly encouraging in that the proportion of clients who reported that they 
"almost never" read with their young children declined from approximately 19 
percent at intake to about 8 percent at follow-up. 

Clients' Reasons for Terminating Service 

Because many clients leave abruptly, local program staff could not provide a 
reason for client termination in 70 percent of the cases. However, as part of the 
telephone follow-up survey, we directly asked the clients why they left adult 
education. Responses to these open-ended questions were content-analyzed, coded 
into categories, and summarized in exhibit 4.20. Subcategories total to more than 100 
percent because some clients gave more than one reason; the five main categories add 
to 100 percent. 
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Exhibit 4.20 
Reasons Clients Left the Program 
(N = 4,653) 



Reason for Leaving 


Percent of Clients 


ESL 


ABE 


ASE 


Overall 


Left satisfied 


29 % 


41% 


54 % 


41 % 


Completed program 


22 


29 


39 


30 


Completed required attendance 


1 


2 


3 


2 


Got what went for/achieved goals 


2 


4 


6 


4 


Other (e.g., enrolled in school, got a job) 


6 


8 


8 


7 


Outside events 


57 


42 


34 


45 


Personal Ulness, health problems 


5 


5 


4 


5 


Family responsibilities (includes child care) 


14 


12 


9 


11 


Transportation problems 


5 


5 


3 


4 


Change of work/job responsibilities 


28 


14 


13 


19 


Other reasons not to do with course itself 


9 


9 


7 


8 


Instructional factors 


6 


9 


6 


7 


Personal embarrassment/ discomfort 


1 


1 


0 


1 


Lack of progress/dissatisfaction w program 


1 


2 


2 


2 


Took too much time and energy (e.g., too far) 


3 


4 


4 


4 


Other reasons reflective of the program 


4 


4 


5 


4 


Combination of above categories 


6 


4 


4 


4 


No reason given (can't say, just left) 


2 


4 


2 


3 


Total 


100 % 


100 % 


100 % 


100 % 



A plurality of clients (45 percent) indicated that they left adult education for 
reasons external to the program. Such nonprogram-related reasons for leaving adult 
education are particularly characteristic of ESL and ABE clients. Most often, ESL and 
ABE clients cited employment-related or family-related reasons for leaving adult 
education. Overall, only 7 percent of the clients left adult education for instructional 
reasons which suggest that they were dissatisfied in some way with the program. 

ASE clients most often left adult education because they had completed their 
course of study. Altogether, 41 percent of the clients indicated they had left the 
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program satisfied, most frequently because of program completion. Because many 
clients appear to behave in terms of the traditional school year, "completing the 
program" may simply indicate that the semester had ended. 

To assess the relationship between attendance and reasons for leaving the 
program, we looked at the number of hours attended from intake to departure and 
the number of weeks enrolled. Exhibit 4.21 shows the median number of hours and 
the median number of weeks attended for clients in each reason-for-leaving category. 
From the standpoint of total (median) hours of instruction, the data in exhibit 4.21 
indicate that clients in each instructional component who were satisfied with their 
program of instruction attended adult education classes longer than clients who left 
the program for reasons other than being satisfied. On the other hand, clients who 
were dissatisfied with their program of instruction stayed in adult education the least 
amoimt of time. 

Exhibit 4,21 

Reasons for Leaving Associated with Two Measures of Attendance 
By Instructional Component 
(N = 4,653) 



Measures of Attendance 
by 

Component 


Reason Client Left Program 


Satisfied 


Outside Events 


Because of Problems 
with Instruction 


Other Reasons 


ESL 


Median hours 
Median weeks 


83 
13 


51 
9 


35 
8 


41 
9 


ABE 


Median hours 
Median weeks 


45 
12 


27 
10 


24 
7 


34 
8 


ASE 


Median hours 
Median weeks 


38 
10 


23 

7 


20 
6 


20 
5 



Overall Benefits Relative to the Purposes of the Act 

Based on the preceding analyses of client outcomes related to basic education 
skills, employment, and continuing education, we provide estimates in exhibit 4.22 
of the aggregate proportion of former clients who indicated that they benefitted from 
participation in adult education with respect to the three purposes of the Adult 
Education Act. The summary data in the top half of the exhibit, pertaining to each of 
the three purposes of the legislation, show that the federal Adult Education Program 
benefits clients primarily in the area of improving basic education skills. Indeed, as 
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indicated by the bottom half of the exhibit, most clients benefit from adult education 
primarily in one of the three areas specified by the legislation, and for most this is in 
the area of basic education skills. While a quarter of all participants did benefit from 
program participation with respect to at least two of the purposes specified by the 
legislation (most often improvements in basic education skills and employability), it 
was rare for clients to benefit in all three areas targeted by the Adult Education Act. 

Exhibit 4.22 

Proportion of Clients Benefitting in One or More of the Ways 
Identified in the Legislation 
(N= 4,653) 



Legislation Puq)ose 


Percent of Clients Benefitting 


ESL 


ABE 


ASE 


Overall 


Basic education skills 


60 % 


61 % 


53% 


58% 


Employment 


35 


23 


24 


29 


Continuing education 


4 


5 


11 


7 


Benefitted in at least one purpose of Act 


75 


71 


66 


70 


Benefitted in at least two purposes of Act 


27 


25 


23 


25 


Benefitted in all three purposes of Act 


1 


1 


1 


1 



In exhibit 4.23, clients in the telephone survey who benefitted from program 
participation in at least one of the three areas specified in the legislation are 
compared with clients who did not achieve any benefits from participating in adult 
education; the comparison is based on median hours of instruction and median 
weeks of enrollment. For all three instructional components, the data show that the 
amoimt of instruction is greater for clients who benefitted from program participation 
compared to their counterparts who did not achieve benefits from participation. 
However, one can not tell from these data whether clients benefitted from 
participation because they received a greater amount of instruction or whether those 
who benefitted simply elected to continue their participation longer for whatever 
reason. In addition, tiie reader should keep in mind that the telephone survey data 
tend to underestimate the number of hours of instruction clients received in adult 
education. 
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Exhibit 4.23 
Relationship of Amount of Instruction 
To Reported Benefits of Participating in Adult Education 

{N = 4,653) 



Composite Measure of One or More 
Benefits: 
Basic Skills, Employment, 
Continuing Education. 


Median Hours of 
Instruction 


Median Weeks of 
Instruction 


Benefitted 


No Benefit 


Benefitted 


No Benefit 


ESL Component 


60 


48 


10 


9 


ABE Component 


45 


20 


12 


6 


ASE Component 


36 


20 


10 


6 


Overall 


48 


27 


10 


7 



Summary and Conclusion 



This chapter has presented findings based largely on client perceptions as 
derived from the telephone follow-up survey with respect to the three central 
purposes of the Adult Education Act: the acquisition of basic education skills; 
enhanced employability; and enabling clients to continue their education to at least 
the completion of secondary school. 

A principal finding in the area of basic education skills was the perception of 
a majority of clients that they had been helped to improve ''a lot" in at least one area 
of basic skills, A related finding from the telephone survey was that clients who 
perceived themselves as having benefitted from adult education in the area of basic 
skills tended to have had a greater amoimt of instruction compared to clients who 
perceived themselves as not having benefitted from basic skills education. We also 
reported that advances in instructional placement level were greatest for beginning 
students. 

There was a net increase of almost 10 percent in employment following 
participation in adult education. ESL clients reported having benefitted the most in 
this respect. Compared to ABE and ASE clients, a significantly greater percentage of 
former ESL clients reported having benefitted from adult education with respect to 
becoming employed or finding additional employment after leaving adult education. 
For clients who were employed both prior to enrollment and subsequent to leaving 
adult education, a significantly greater proportion of ESL clients compared to ABE or 
ASE clients reported that participation in adult education had helped to improve 
their job performance or had helped them to acquire a better job than they had prior 
to enrolling in adult education. 
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The telephone survey results indicated that approximately 17 percent of the 
former adult education clients are continuing their education, with 7 percent of the 
program's participants now enrolled in post-secondary education. We estimated that 
between 11 and 28 percent of the program's ASE clients (or, 4 to 12 percent of all 
clients served) were helped by adult education to complete their secondary 
education. 

In addition to goals related to employment and basic skills, most adult 
education clients also entered the program with personal goals related to the 
enhancement of self-esteem or self-concept. The telephone survey data suggest that 
adult education was successful in helping approximately 70 percent of the program's 
participants who had such personal goals to improve their self-concept. The 
telephone survey data also indicated a net increase of 15 percent in the number of 
former clients who read more frequently to yoimg children in their household than 
they did prior to enrollment in adult education. 

While 41 percent of the participants left adult education after having 
completed their program of study in one fashion or another, a plurality of clients (45 
percent) left for reasons external to the program. More specifically, ASE clients were 
most likely to have left adult education because of program completion whereas ABE 
and ESL clients were most likely to have left the program because of employment 
and family-related responsibilities. Approximately 7 percent of the clients terminated 
instruction because of dissatisfaction with their adult education program. 

Approximately 70 percent of the clients are estimated to have benefitted from 
participation in adult education with respect to the purposes of the Adult Education 
Act. The majority of clients (58 percent overall) perceive these benefits to be in the 
area of improving basic education skills. 
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Chapter 5 

PROGRAM RESULTS 



The central purpose of the National Evaluation of Adult Education Prograir.s 
was to assess the potential of programs supported by the Adult Education Act "for 
significantly reducing deficits in the adult population with respect to literacy, English 
proficiency, and secondary education." In previous chapters of this report v%^e have 
shown that: 

• The program .erves approximately 1.7 million new clients every year. 

• Clients that began instruction receive a median of about 58 hours. 

35 hours for ABE. 
28 hours for ASE. 
113 hours for ESL. 

• Most of the clients represented by our telephone follow-up survey 
report that they benefitted from their participation in some meaningful 
way; some objective indicators of client level outcomes tend to support 
the clie'^t's self-reports. 

In this chapter we relate estimates of the size of the program's target 
population to information on client enrollment and participation. As these analyses 
show, the program serves substantially more ESL clients per 1,000 members of the 
target population than members of the other two subgroups (i.e., ABE or ASE). They 
also show that, depending on the time of year, between 16 and 25 percent of 
programs report that they have waiting lists of clients they are imable to serve, with 
most of the clients on these lists waiting to begin instruction in ESL. Finally, the 
analyses snow that the program's target population increases by more than twice as 
many persons each year as the program is successfully able to serve. 

Target Population Estimates 

Target population estimates for Adult Education Programs were completed by 
Research Triangle Institute using data from the 1990 Census of Population and 
Housing (CPH) [Thorn and Fleenor 1993].^^ The 1990 census has the most 
appropriate data for making these estimates because these data are the only national 



RTI used the Public Use Microdata Samples (PUMS) data base, which contains 1990 census 
information on educational attainment and English proficiency for a 5 percent sample of the 
households in the country, with sampling weights used to produce estimates for the total United 
States population. 
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data available that match the service areas of the adult education programs. The 
target population is defined as persons 16 years old and older who have not 
completed secondary school, have not received a GED, and are not currently enrolled 
in school. About one-fourth (27 percent) of the total adult population in the United 
States falls into this category. 

RTI developed national, regional, and state profiles of the target population, 
providing estimates by educational attainment, proficiency in English, age, 
race/ethnicity, and other characteristics. In summary, their findings are as follows: 

• Some 39 percent of the target population are located in the South, 22 
percent in the North Central region, 20 percent in the Northeast, and 19 
percent in the West. 

• A large proportion of the target population (60 percent) have completed 
9 to 12 years of school; the remaining 40 percent have less than a 9th 
grade education. 

• Within the target population, 23 percent speak English as a second 
language. 

• Approximately 41 percent of the target population are 60 years of age or 
older, 28 percent 25 to 44, 20 percent 45 to 59, and 11 percent 16 to 24. 

• Estimates by racial/ethnic groups indicate that 67 percent of the target 
population are white, 16 percent black, 13 percent Hispanic, 2 percent 
Asian/Pacific Islander, and less than 2 percent American 

Indian /Alaskan Native. 

Target population estimates by region and educational group are presented in 
exhibit 5.1. As noted, these estimates exclude all adults who have attained a high 
school diploma or the equivalent. The Adult Education Act, however, permits 
serving these adults if they meet other criteria of need. Adults whose first language 
is not English and who do not speak English very well accoimt for more than 45 
percent of adult education program clients; about half of these ESL clients have 
attained at least a high school diploma. Therefore the estimates presented here 
underrepresent the number of ESL adults who are eligible for adult education 
programs, and the ASE estimate of the target population does not include adults who 
are native English speakers with a high school diploma but low reading skills. 

The data show that the largest target population is the adult secondary 
education population-adults who have completed 9 to 12 years of school but have 
not received a diploma or its equivalent. This is true for the United States as a whole 
and for three of the four regions. In the West, the ESL target population is the 
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largest, but about the same size as the ASE target population for that region (9 
percent of the total target population compared with 8 percent). 



Exhibit 5-1 

Estimates of Target Population by Region and Instructional Component 



Region 


Adult Education Target Population 


ESL 


ABE 


ASE 


Total 


USA 


10,179,379 


11,545,723 


22,341,013 


44,066,115 


North Central 


1,147,486 


2,883,504 


5,515,358 


9,546,348 


Northeast 


2,523,607 


1,910,890 


4,303,534 


8,738,031 


South 


2,672,517 


5,618,931 


9,145,264 


17,436,712 


West 


3,835,769 


1,132,398 


3,376,857 


8,345,024 



Note: The target population estimates exclude all adults age 16 and older who have a high school 
diploma or the equivalent or who are enrolled in school. The Adult Education Act permits serving 
these adults if they meet the criteria of need. 



To more accurately reflect the target population, we included ESL adulcs with 
a diploma in our calculations who reported that they speak English less than 'Very 
well" This has the effect of increasing the number of ESL adults in the target 
population from 10,179,379 (23 percent of the total) to 12,322,835 (27 percent of the 
total). Thus the total target population increases from about 44.1 million to 46.1 
million adults. Exhibit 5.2 compares the ESL target population estimates with and 
without a high school diploma or the equivalent. 



Exhibit 5.2 

Estimates of Target ESL Population by Region 



Region 


Target ESL Population* 


ESL without a Diploma 


ESL with a Diploma 


Total ESL 


USA 


10,179,379 


2,143,456 


12,322,835 


North Central 


1,147,486 


244,547 


1,392,033 


Northeast 


2,523,607 


624,317 


3,147,924 


South 


2,672,517 


543,884 


3,216,401 


West 


3,835,769 


730,708 


4,566,477 



* ESL refers to those adults who speak English as a second language and who indicated in the 1990 
Census that they do not speak English "very well." 
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The addition of adult ESL speakers with a diploma to the total target 
population changes the proportion of ABE eligibles in the target population from 26 
percent to 25 percent and reduces the proportion of ASE eligibles from 51 percent to 
48 percent. Their relative positions do not change, however; the total ASE target 
population is still the largest and the ABE target population is still comparable to the 
ESL population. 

Rates of Participation 

To calculate participation rates, we used client intake records obtained from 
our national sample of 22,548 clients who enrolled in adult education programs 
between April 1991 and April 1992. Exhibit 5.3 presents by region and instructional 
component the number of active clients who had no high school diploma or the 
equivalent (a subset of all newly enrolled ABE and ASE clients) plus the total number 
of clients in ESL. As the exhibit shows, the number of clients is much larger within 
the ESL component than it is within ABE or ASE components. Also, the numbers of 
new clients vary by region, with the largest concentration of new clients in the West, 
followed by the South, the North Central region, and the Northeast. The new ESL 
clients are heavily concentrated in the western states, while ABE and ASE new clients 
are heavily located in the South. 

Exhibit 5.3 

Number of New ABE anci ASE Clients Without High School Diploma or the 
Equivalent, and the Number of New ESL Clients 
By Region and Instructional Component 



Region 


Number of Clients 


ESL 


ABE 


ASE 


Total 


USA 


823,300 


274,200 


387,100 


1,484,600 


North Central 


69,500 


79,200 


102,600 


251,300 


Northeast 


41,200 


37,500 


Jl,900 


130,600 


South 


98,700 


139,000 


162,700 


400,400 


West 


613,900 


18,500 


69,900 


702,300 



Note: New-client estimates exclude ABE and ASE clients with a high school diploma or the 
equwalent. 
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Rates of Participation by Region and Instructional Component. Using target 
population estimates that include all ESL adults who do not speak English very well 
and the number of active first year clients, we calculated the United States and 
regional participation rates overall and for each instructional component.^^ Exhibit 
5.4 presents this information. 

The rate of participation is the number of new clients for every 1,000 eligible 
persons. For example, the total new-client rate for the Uxiited States is 32, meaning 
that for every 1,000 persons who are eligible for adult education in the United States, 
32 enroll and receive at least 1 hour of instruction as new clients in adult education 
programs in a given year. In general, the exhibit shows the highest rates of new 
clients in the ESL component and the lowest in the ASE component. Regionally, 
rates of participation are highest in the West and lowest in the Northeast. New-client 
rates across the United States range from a low of 12 persons for every 1,000 eligible 
for ASE in the Northeast to a high of 134 for ESL in the westem United States. 

About 75 percent of the ESL target population in the United States is located 
in the West, id about 87 percent of the West's target population is eligible for ESL 
(see exhibit 5.3). This compares to the data reported in chapter 2 indicating that 
some 82 percent of the clients in the West are enrolled in ESL and that the westem 
region accounts for about 72 percent of all ESL clients enrolled in the program. For 
ABE, on the other hand, the participation rate in the West is considerably less than 
elsewhere, and ABE clients are only about 4 percent of the clients enrolled in that 
region. 



^® RTI target population estimates for ABE and ASE populations included persons eligible for 
ESL. Our calculations treat the ESL population separately. 
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Exhibit 5.4 

Participation Rates in Adult Education by Region 
and Instructional Component 



Region 


Rates of Participation 


ESL 


ABE 


ASE 


Total 


USA 


66 


23 


17 


32 


North Central 


49 


27 


18 


25 


Northeast 


13 


19 


12 


13 


South 


30 


24 


17 


22 


West 


134 


16 


20 
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The data also show that the proportion of the ESL target population being 
serv^ed is substantially greater in the West than in other regions, and that ABE clients 
in the West are underserved in comparison with the rest of the United States. As 
exhibit 5.5 shows, a member of the ESL target population in the West is some eight 
times as likely to be served by the program as an adult in the ABE target group. 



Exhibit 5.5 

Ratios of Target Population Participation Rates by Region 



Region 


Ratios of Participation Rates' 


ESL/ABE 


ESL/ ASE 


ABE/ASE 


North Central 


1.8 




2.7 




1.4 


Northeast 


0.7 




1.1 




1.5 


South 


1.3 




1.8 




1.6 


West 


8.4 




6.7 




1.4 


U.S. Total 


2.9 


3.9 


1.4 



' Participation rate = Number of new clients served over 1 year period per 1,000 in the 
target population. See exhibit 5.4 for actual rates. 
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In the remainder of this section we present rates of participation by age and 
by race /ethnicity for the United States and the four census regions. Because data on 
the number of ESL clients with diplomas by age or by race /ethnicity were not 
available, the participation rates we report in exhibits 5.6 and 5.7 are based on a 
smaller target population number (i.e., 44 million rather than 46 million). In order to 
compare equivalent populations, we excluded ESL adults with a diploma or the 
equivalent from our new-client enrollment data for these analyses. Because the 
proportion of new ESL clients with a diploma is larger than the proportion of ESL 
with a diploma in the target population, participation rates presented by age and 
racial /ethnic groups are lower tiian they would be if these ESL adults were included. 

Rates of Participation by Age. The target population estimates reported by 
RTI indicate a difference in eligibility by age. More than 40 percent of the adults 
eligible for adult education program services are at least 60 years old. Adults in this 
age group have the lowest educational attainment; 53 percent have less than a 9th~ 
grade education. In contrast, adults 16-24 years of age constitute only 11 percent of 
the total target population, and 20 percent of them have less than a 9th-grade 
education. Client enrollment data indicate that the average age of new clients is 31 
years. Some 3 percent of clients are 60 years of age or more, and 43 percent are 
younger than 25. Only 9 percent of new clients are over 45 years of age. 

We calculated participation rates for each age group (16-24, 25-44, 45-59, 60 
and over).^^ These data are presented in exhibit 5.6. The highest participation rates 
are among the youngest age group, 16-24. Participation rates steadily decrease as the 
age of the target population increases, with rates among the people age 60 .and older 
being very low. From these data we can conclude that the older target population is 
not being proportionately served. 

As exhibit 5.6 shows, participation rates are highest in the West across all age 
groups. These data are not surprising because the western states, which include 
Califomia and Texas, have a large population of normative speakers of English. As 
noted, almost half of ail program clients speak English as a second language. 



We use these age categories, rather than those found elsewhere in the report, to correspond 
with the available census data categories. 
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Exhibit 5*6 
Rates of Participation in Adult Education, 
By Region and by Age Group 



Region 


Rates of Participation 


16-24 


25-44 


45-59 


60 + 


USA 


83 


30 


7 


1 


North Central 


101 


34 


7 


1 


Northeast 


57 


22 


4 


<1 


South 


59 


17 


4 


1 


West 


122 


54 


7 


5 



Note: Rates of participation among 1,000 eligible persons for adult education programs in 
1990, excluding all adults age 26 and older who have a high school diploma or the equivalent 
or who are enrolled in school. 



Rates of Participation by Racial/Ethnic Group. Because different racial or 
ethnic groups may have different rates of nev^-client enrollment, we calculated the 
rate of nev^-client enrollment across five groups (American Indians/Alaskan Natives; 
Asians/Pacific Islanders; non-Hispanics blacks, Hispanics; and non-Hispanic 
whites). As exhibit 5.7 shows, Asian/ Pacific Islander adults are the most likely to 
participate in adult education programs, followed by Hispanic adults. There is 
regional variation, with Asians or Pacific Islanders more likely to participate in the 
North Central region, Hispanics more likely to participate in the South, and American 
Indians /Alaskan Natives more likely to participate in the Northeast or West. 
Overall, non-Hispardc whites are proportionately the least likely to participate in 
adult education programs. 



RTI target population estimates by race /ethnicity have five Hispanic and five non-Hispanic 
categories. Our client enrollment data have only the five categories of race/ethnicity just listed. For 
comparative purposes, we combined all RTI Hispanic categories into one group. One RTI category, 
"other, non-Hispanic" had to be omitted from our calculations of participation rates because our client 
enrollment data have no equivalent group. 
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Exhibit 5.7 

Rates of Participation, by Region and by Racial/Ethnic Group 





Racial /Ethnic Group 


Reffion 


American Indian 
or Alaskan Native 


Asian or Pacific 
Islander 


tJiack, not 
Hispanic 


Hispanic 


wnite, not 
Hispar'c 


USA 


52 


64 


16 


55 


13 


North Central 


47 


177 


17 


66 


18 


Northeast 


71 


12 


14 


31 


10 


South 


8 


23 


16 


14 


13 


West 


80 


76 


16 


76 


11 



Note: Rates of participation among 1,000 eligible persons for adult education programs in 1990, 
excluding all adults age 16 and older who have a high school diploma or the equivalent or who are 
enrolled in school. 



Cunent Capacity to Enroll Additional Clients 

Another study objective was to explore the relationship between the demand 
for program services and the capacity of adult education programs to meet that 
demand. One of the measures we considered was the proportion of programs with 
clients on waiting lists. Programs in our survey of all local providers (Universe 
Survey; items 20, 22) were asked to report the number of clients on waiting lists at 
the end of June 1990 and at mid-October 1990. From these responses, we found that 
16 percent of programs reported having clients on waiting lists at the end of June 
1990, and 25 percent of the programs had clients on their waiting lists in mid-October 
1990. These programs with waiting lists may be considered to have demand for 
adult education services in excess of their capacity to provide services. 

Using this same information, we also compared the aggregate and the mean 
number of clients on waiting lists within the three instructional components (ESL, 
ABE, ASE). As shown in exhibit 5.8, our data indicate that the average number of 
ESL clients on waiting lists at the end of June 1990 or the middle of October 1990 
was larger than the average number of ABE or ASE clients for the same two periods. 
It appears that ESL components are more likely to have an unmet demand for 
services than either ABE or ASE components. 
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Exhibit 5.8 

Number of Clients on Waiting List, by Instructional Component 

(N = 112) 





Total Number of Clients on 
Waiting Lists 


Mean for Programs with 
Waiting Lists 




ESL 


ABE 


ASE 


ESL 


ABE 


ASE 


June 1990 


44,800 


16^00 


8,100 


313 


61 


51 


Mid-October 1990 


40,800 


13;200 


9,600 


86 


53 


30 



As part of our analysis of demand and capacity, we also examined program 
responses to an item on the survey of all providers (Universe Survey, item 23) vy^hich 
asked program directors to provide the number of additional clients, by component, 
whom they could have accommodated if more people had shown up for instruction. 
Programs reporting that one or more additional clients could be served were 
considered to be operating below capacity; approximately 59 percent of all programs 
fit this definition of having capacity to spare. 

We also compared the mean number of additional clients who could be served 
wathin each of the three instructional components. Most programs reported that they 
could serve fewer additional ESL clients (33 on average) tiian ABE clients (44) or ASE 
clients (47). These results suggest that ESL components are less likely to be operating 
below capacity than ABE or ASE components. 

Exhibit 5.9 provides estimates for mid-October 1990 of the number of clients on 
waiting lists, the number of additional clients the program directors with waiting lists 
said they could currently be serving if the clients had shown up at the right time, the 
number of additional clients who could have been served by programs with or 
without waiting lists, and the number of clients they said were currently enrolled. 
As may be seen from the exhibit: 

• Across all programs, the number of additional clients that directors say 
their programs could be serving is about 10 times greater than the 
number on waiting lists. 

♦ Overall, the number of clients waiting to be served is about the same as 
the amoimt of the reported excess capacity in programs with waiting 
lists. However, within components, there are more ESL clients waiting 
to be served than there is excess capacity in programs with waiting lists. 
The opposite is true for ABE and ASE. 
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• Programs report that they could be serving almost half again (42 
percent) more clients than they currently enroll. 

While these data clearly indicate that the system has the capacity to serve additional 
clients, the extent to which accessing that capacity is dependent on additional 
landing is not clear. Responses from some program directors to the question about 
their ability to serve additional clients suggest that their estimates assumed some 
additional resources would be forthcoming. 



Exhibit 5.9 

Number of Clients on Waiting Lists and the Number That Programs Could Have 

Served and Were Serving 



Number of Clients as of Mid-October 1990 


ESL 


ABE 


ASE 


Total 


On waiting lists 


40,800 


13;200 


9,600 


63,600 


Others who coiild have been served by 
programs with waiting lists 


28,000 


21,500 


12300 


61,800 


Others who could have served by all 
programs (with or without waiting lists) 


141,700 


361,600 


138,400 


641,700 


Total reported as being served 


537,200 


454,000 


533,500 


1,524,700 



To explore this topic further, we also asked the programs responding to the 
Program Profile (item 4) to report the maximum number of clients in each program 
component who could be served at any one time. Because a number of factors— 
assumptions about fimding availability, qualified staff, and instructional space— can 
influence estimates of program capacity, responses to maximum enrollment items 
may reflect assumptions about future or possible capacity rather than current reality. 
For example, several programs reported that there was no maximum to their 
enrollments and that they could be expanded to meet local adult education needs. 

To determine which programs were operating below, at, or above capacity, we 
computed the ratio of current enrollment figures reported for mid-October 1990 (item 
3) to the maximum enrollment reported for each program. Levels of program 
capacity are defined as follows: a program is operating below capacity if the ratio of 
current to maximum enrollment is between 0 and 0.63. A program is operating at 
capacity if the ratio of current to maximum enrollment is between 0.64 and 1.5.^^ 



Programs were considered to be "at capacity" if the ratio of current to maximum enrollment was 
within one-half of a standard deviation from tlie mean. 
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Programs in which the ratio of current to maximum erurollment exceeds 1.5 are 
considered to be above capacity. 

Using this definition we found the following: 

• 23 percent of the programs are operating below capacity; 

• 63 percent are operating at capacity; and 

• 14 percent are operating above capacity. 

Exhibit 5.10 presents the level of capacity within ESL, ABE, and ASE 
components. ABE components are the least successful at balancing program demand 
and capacity, and ESL components are the most successful. Approximately 43 
percent of the ABE and 24 percent of the ASE components are operating below 
capacity, compared with only 8 percent of the ESL components. These findings 
correspond with data from the Universe Survey. Programs with an ESL component 
are least likely to have underutilized capacity. 

ESL prograips are also less likely than ABE and ASE programs to have 
seriously exceeded their capacity. Most (87 percent) of the ESL components are 
operating at capacity, compared with 42 percent for ABE components and 63 percent 
for ASE components. Relatively few programs (between 5 and 16 percent) have 
components that are operating substantially above this program capacity indicator. 

Exhibit 5.10 

Utilization of Program Capacity by Predominant Component* 

(N = 95) 



Capacity 


Percent of Programs 


ESL 


ABE 


ASE 


Below 


8 % 


43 % 


24 % 


At 


87 


42 


63 


Above 


5 


16 


13 


Total 


100 % 


100 % 


100 % 



* Defined as the component serving the largest number of clients. 
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We also related this measure of utilization of program capacity to measures of 
program size, the urbarucity of the program's service area, the type of sponsoring 
agency, the amount of program emphasis on outreach and recruitment activities, and 
measures of average class size and hours of instruction. In brief we foimd: 

• Little relationship between operating at capacity and program size, 
defined as the number of clients served. Between 43 percent and 79 
percent of programs of any size were operating at capacity. The largest 
programs (1,000 or more clients served) were somewhat more likely to 
be operating at capacity, and the very largest (5,000 clients or more) 
were the most likely to have substantially v'^xceeded the capacity of their 
program (26 percent of these very large programs). 

• Few programs in metropolitan areas (11 percent) operating below 
capacity. There were no programs in large cities in major metropolitan 
areas operating below capacity. However, in nonmetropolitan areas, 42 
percent of programs indicated they were operating below capacity .^^ 

• Programs operated by community colleges most likely to be operating 
below capacity. This was the case for some 59 percent of community 
colleges, 28 percent of programs operated by public school districts, and 
19 percent of programs operated by other service providers. 

• No relationship between average class size or average hours of 
instruction and capacity. 

• A relationship between outreach efforts and below-capacity operation, 
but not the one we expected. We foimd that no programs with high 
outreach effort were operating above capacity and 57 percent were 
operating below capacity. Among programs with a low or medium level 
of outreach effort, more than 55 percent are operating at capacity 
compared with 43 percent among programs with a high level of 
outreach effort.^^ 



Community designations are based on 1990 Census information plus responses to the Universe 
Survey on type of community served. Major metropolitan areas are defined as those having a 
population of 1.5 million or more; large cities as those having a population of 500,000 or more, and 
small metro areas as any community located within a standard metropolitan area with a population of 
less than 1.5 million. 

Program Profile items were used to develop a composite measure of the extent to which 
progreuns emphasize program recruitment/outreach activities. Programs were assigned an outreach 
effort score ranging from 0 to 10. Scores were then collapsed into low (0-1), medium (2-4), oi high (5 
or more). 



79 



Comparisons of Program Successes with New Additions to the Target Population 

A central purpose of this study was to assess the potential of programs funded 
under the Adult Education Act to significantly reduce the literacy needs of the target 
population. Iii this section we compare estimates of the annual additions to the 
target population with estimates of the number of clients who received sufficient 
hours of instruction to be counted as program successes, or subtractions from the 
program's target population. By focusing on this comparison, we are being 
consistent with the data collected through this study and we avoid essentially 
unanswerable questions about the impact of past program activities. 

It should be noted that we are not estimating net annual changes in the 
program's target population. Specifically, subtractions attributable to mortality, the 
efforts of adult education programs not receiving federal funding, and literacy gains 
achieved by wholly informal means are not considered in our analysis.^ 

Annual Additions to the ABE, ASE, and ESL Target Population. Estimating 
the annual addition to the program's target population involves using census data 
and making a number of steady-state assumptions (e.g., no dramatic changes in the 
patterns of school completion or the immigration of non-English speakers in the past 
several years). A summary of our estimates is presented in exhibit 5.11. As the 
exhibit shows, overall, we estimate that additions to the target population for the 
adult education program amount to about 1.5 million persons a year, with the annual 
increase being the greatest for ESL (643,000), followed by ASE (590,000) and then ABE 
(214,000). 



or more). 

^ Mortality is probably the largest annual subtraction from the ABE target population, since 22 
percent of the 17.4 milUon adults with less than 9 years of education are age 75 or over. Annual 
mortality in this group amomits to about 8 percent, or about 312,000 deaths each year. (March 1993 
Current Population Survey Report on Educational Attainment, in Statistical Abstract of the United States: 
U.S. Death Rates by Age). 
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Exhibit 5.11 

Estimated Annual Additions to the Target Population 



Instructional 
Component 


Basis for Estimate 


Number of 
Adults 


ASE addition' 


Number of new high school dropouts plus 
number of new high school graduates not 
continuing their education and estimated to 
read below an 8th grade level. 


590,000 


ABE addition^ 


Number of new grade school dropouts, plus 
allowance for immigrants of all ages with no 
high school education who have just gained 
sufficient proficiency in English to benefit 
from ABE instruction. 


214,000 


ESL addition' 


Number of all who immigrated to U.S. from 
non-English speaking countries. 


643,000 


Total 


Sum of ASE, ABE and ESL 


1,447,000 



* The average number of high school dropouts per year for 1990-1992 was 359,000 
(NCES Dropout Rates in the United States: 1992). For the same period the U.S. 
Department of Education reports the average number of high school graduates per 
year was 2.26 million. According to the 1985 Young Adult Literacy Survey (Kirsh, et 
al.), 40 percent of high school graduates do not go on to college, and 27 percent of 
these graduates read below an 8th grade level (231,000). 

^ Estimated from census data to be about 109,000 new grade school drop outs, plus 
about l/15th of the total number who speak a non-English language at home and 
completed less than 9 years of school and report speaking English less than "very 
well" (l/15th of 1,573,000 = 105,000). 

The average number of persons who immigrated to the U.S. per yecir during the 5 
years prior to the 1990 census, excluding those who were under 18 years old, is 
699,000; the proportion of immigrants to the U.S. from non-English-speaking 
countries between 1980 and 1990 was 92 percent (U.S. census data). 

Estimating the Number of Program Successes. Establishing a basis for 
estimating the number of clients who received sufficient instructional services to have 
had their literacy needs significantly reduced is problematic. The data available from 
the national evaluation do not provide as clear a basis as one might wish. When the 
study was designed, the plan was to use the results of the analyses of reading 
achievement tests as the basis for this analysis. As discussed m Chapter 3, however, 
programs were generally unable to provide valid pre- and posttest scores on clients 
participating in the study, and analyses of the relatively small number of achievement 
test scores do not support their use as indicators of literacy-needs reduction. 
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Consequently, we considered two alternative indicators of client outcome: the 
time required for instructional advancement, and the time required to achieve self- 
reported accomplishment of goals. 

Instructional advancement: Early in the national evaluation. Department of 
Education representatives on the study's technical advisory panel suggested 
that the proportion of clients who move from one instructional level to another 
be considered an important indicator of program performance.^^ As part of 
the national evaluation, we obtained information on each client's initial 
instructional placement, their instructional placement at the close of each study 
reporting period (every 6 to 8 weeks) and tiKe number of hours of instruction 
each client received each week in which they were enrolled. 

Using advancement in instructional placement as an indicator of reduced 
literacy needs requires deciding how far clients should advance. For ABE and 
ESL, there are two reasonat e alternatives: 

• Moving up one instructional level — for ABE this is going from the 
beginning level to the intermediate or from the intermediate to ASE; for 
ESL it is going from the beginning level to intermediate or from the 
intermediate level to advanced. 

• Attaining an advanced level which meets the intention of the Act — for 
ABE that is going to ASE; for ESL it is going from either beginning or 
intermediate to advanced. 

For ASE and most clients in advanced ESL, there is no higher instructional 
level, so an alternative measure is required. For ASE it is reasonable to use an 
indicator of program completion, because the adult education program's 
instruction terminates witibi high school completion or its equivalency. Since 
our telephone follow-up survey provides information on the reasons clients 
terminate instruction, a feasible indicator of program completion for ASE is the 
respondents' self-report that they left instruction satisfied because they had 
"completed the program." For clients who enrolled in the ESL-Advanced level, 
however, there is no comparably clear end point. Since it can be reasonably 
argued that the purposes of the Act with respect to citizenship and 
employability require that all ESL clients attain at least the level of advanced 
ESL, but that requirements thereafter are dependent entirely on the needs of 
each individual, we decided to count all clients who received instruction at the 
advanced level of ESL as program successes. 



The federal adult education reporting system collects data on the number of clients in each state 
who enroll at each instructional level complete that level, separate before completing that level, and 
move to a higher level. 
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Exhibit 5.12 shows the movement of new clients with one or more hour of 
instruction from their initial instructional placement to their placement at the 
time they left the program.. As the exhibit shows, 59 percent of the clients 
who began instruction at the beginning ESL level remained at that level of 
instruction, while 28 percent moved to the intermediate level, and 12 percent 
had moved to the advanced level by the time they left the program. As these 
data show, about 1 percent of the ESL-Beginning, 5 percent of the ESL- 
Intermediate, and 12 percent of the ESL-Advanced clients had moved to ABE 
or ASE instruction by the time they exited the program. Thus, the target 
population for ABE and ASE is augmented each year by about 3 percent of the 
clients whose initial instmctional placement is in ESL. This amoimts to an 
annual addition of about 16,300 to the ABE/ASE target group. 



Exhibit 5,12 

Placement of ESL and ABE Clients When Leaving by Placement at Time of Entry 



Instructional 
Component 


Percent of Clients 


ESL-Beg 


ESL-Int 


ESL-Adv 


ABE-Beg 


ABE-Int 


ESL-Beginning 


59 % 






0.3 7o 


0.2 % 


ESL-Intennediate 


28 


72 % 




0.7 


0.4 


ESL-Advanced 


12 


23 


88% 


0.7 


0.3 


ABE-Beginning 


0.3 


1 


3 


70 




ABE-Intermediate 


0.1 


2 


5 


18 


78 


ASE 


0.9 


2 


4 


11 


21 


1 Total Number 


438,285 


87,382 


51,478 


70,773 


114,251 



To operationalize these indicators, we calculated the median number of hours 
clients received instmction between the time of their initial hour of instruction 
and the end of the study reporting period in which they advanced. This 
results in a somewhat generous estimate of the number of successes, since the 
time needed for clients who had not advanced by the end of the study could 
not be included in the calculations.^ Clients who went backward (e.g., from 
intermediate to beginning levels) were excluded from the analyses because we 
assumed they were initially misplaced. To determine the number of new 
clients to which these hour thresholds should apply, we treated each 



^ Since the calculations are based on clients who did advance, only about 25 percent of ABE and 
37 percent of ESU and the time needed for clients who had not advanced by the end of the study 
could not be included in the calculations, they provide an underestimate of the number of hours 
needed by most clients to advance. 
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instructional group separately; that is, we calculated the number of new clients 
in each level, and applied the appropriate number of hours for that group. 
The median number of hours to advance by instructional level for ABE and 
ESL clients are shown in exhibit 513. ASE clients who reported they 
terminated instruction because they completed the program received a median 
of 45 hours of instruction. 



Exhibit 5.13 

Median Number of Hours for ABE and ESL Clients to 
Complete 1 or More Instructional Levels 



Instructional Component 


Median Nximber of Hours 
Needed to Complete 


ABE Beginning 


32 


ABE Intermediate 


36 


Beginning and Intermediate ABE 


65 


ESL Begirming 


216 


ESL Intermediate 


136 


Beginning and Intermediate ESL 


372 



Accomplishing self-defined goals: An alternative indicator we considered 
was the median number of hours received by clients who reported that they 
left their instructional program satisfied because they "got what they went for 
or achieved personal goals/' Using this criterion, we find tliat the median 
number of hours of instruction are those shown in exhibit 514. 

It is interesting to note the similarities in the hours shown in exhibits 5.13 and 
5.14. For ABE, the median number of hours to accomplish personal goals and the 
number to advance in placement are fairly similar, especially for clients who entered 
at the intermediate level. For ESL, the number of hours to advance and the number 
needed to accomplish .personal goals are the same for clients at the intermediate level, 
but beginning level ESL students require much longer to advance to an intermediate 
level than they need to meet self-defined goals. 
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Exhibit 5.14 

Median Number of Hours for ABE and ESL Clients to 
Accomplish Self-defined Goals 



Instructional Component 


Median Number of Hours 


ABE Beginning 


51 Hours 


ABE Intermediate 


40 


ESL Beginning 


64 


ESL Intermediate 


136 



Exhibit 5.15 presents our estimated annual impact on the increase in the 
program's target population. Because the language of the Act is focused on assisting 
adults reach at least the secondary school completion level of English language 
proficiency, we have opted to use instructional advancement, rather than personal 
goal attainment, as the basis for these estimates. 

To estimate the size of the aimual increase in the ABE and ESL target 
population by instructional placement level (i.e., beginning, intermediate, or 
advanced), we allocated the estimates of annual additions to the target population 
presented in exhibit 5.11 by the proportion of new-client placements at each sub- 
component level. Of the ABE clients who received one hour or more of instruction, 
42 percent were Lnitially placed in ABE-Beginning and 58 percent in ABE- 
Intermediate. Of the ESL clients with one hour of instruction or more, 74 percent 
were initially placed in ESL-Beginning, 16 percent in ESL-Intermediate, and 10 
percent in ESL-Advanced. 
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Exhibit 5.15 

Estimated Annual Impact on Increases in the Adult Education Target Population 

Based on Program Advancement 



Instructional 
Component and 
Level 


Estimated 
Annual 
Additions to 

Target 
Population 


Number of Clients 
Served by Program 
Who Attained 
Sufficient 
Instrucdonal Hours 
to Reach ASE or 
Advanced ESL^ 


Number of Clients 
Served by 
Program Who 
Attained a 
Sufficient Number 
of Hours to 
Advance One 
Level'' 


Program 
Successes as 
a Percent of 
Estimated 

Annual 
Increases in 

Tareet 
Population 


ABF« Total 


214 000 


123 000 


148 000 


57 to 69% 


/tDH. Deginrung 


Qo nnn 

yUfUyJU 




nnn 


A% kr\ "7 CPU 
*±0 ZO / U /O 


ABE: Intermediate 


124,000 


84,000 


84,000 


68% 


ESL: Total 


643,000 


273,000 


343,000 


30 to 53% 


ESL: Beginning 


476,000 


148,000 


218,000 


31 to 46% 


ESL:Intermediate 
& Advanced 


167,000 


125,000 


125,000 


75% 


ASE 


590,000 


159,000 


159,000 


27% 


Total 


1,447,000 


555,000 


650,000 


38 to 45% 



* Defined for ABE: 65 hours for those entering at ABE-Beginning to move to ASE and 36 hours for 
those entering at ABE-Intermediate to move to ASE. For ESL as: 372 hours for those entering at ESL- 
Beginning to move to ESL-Advanced, and 136 hours for those entering at ESL-Intermediate to move to 
ESL-Advanced. For ASE: for clients to report that they had completed the program (45 hours). 



^ Defined for ABE: for those entering at ABE Begixming to move to ABE-Intermediate (32 hours), and 
for those entering at ABE-Intermediate to advance to ASE (36 hours). For ESL: for those entering at 
ESL Beginning to move to ESL-Intermediate (216 hours), for those entering at ESL-Intermediate to 
move to ESL-Advanced (136 hours). For ASE: for clients to report that they had completed the 
program (45 hours). 

A range of estimates indicating the extent to which the program is successful 
in minimizing the annual increase of the ABE, ASE, and ESL population with English 
literacy needs is presented in exhibit 5.15. The exhibit includes the results of applying 
both criteria for determining instructional advancement (i. e., advancing one 
instructional level or attaining ASE or ESL-Advanced), and where these criteria 
produce different estimates, the table's rightmost column provides the range. As the 
exhibit shows, each year the program's successes amount to less than half the 
estimated additions. Overall, new additions to the program's target population 
exceed relatively generous estimates of program successes by about 800,000. 
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Proportionately, the program's greatest area of impact is in the area of ABE, 
and it is having the least effect on stemming the annual growth in the number of 
potential clients for ASE. As was discussed earlier in this chapter, it should be noted 
that more than 40 percent of the adults eligible for program services are at least 60 
years old and that of this group over 50 percent have less than a 9th grade education 
(i.e., are in the ABE target group). Thus, it is reasonable to conclude that in addition 
to services provided by the adult education program, the size of the ABE target 
population is also diminished considerably each year by the effects of clients' age. 
Looking toward the future, these analyses suggest that it would be appropriate to 
reconsider current legislative and programmatic limitations on the use of federal 
funds to serve clients in ASE. 
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APPENDIX A 
STUDY OBJECTIVES AND WHERE ADDRESSED 



Objective 


Report 
Location 


1. Client populations and patterns of participation. To construct empirically based models of 
client "flows" through each of the program's service components (ABE, ASE, and ESL) which 
will permit detailed estimates of client intake, participation, and attrition over time. 


Rpt 3: ch 2 
Rpt 4: ch 2 


2. Factors contributing to client persistence. To identify client background and service- 
program variables that are positively related to client persistence (or negatively related to client 
attrition). 


Rpt 3: ch 3 
Rpt 4: ch 2, 
ch3&4 


3. Reaching adults with basic literacy needs. To identify service-program characteristics that 
are positively or negatively related to attracting and holding adults with basic literacy needs. 


Rpt 4: ch 4 


4. Support and cooperation at the local level. To assess the extent to which Federal and State 
funds for adult education are effectively supplemented by other resources at the local level. 


Rpt 1: ch 3 
&:4; Cost 
Study Rpt 


5. Program capacity and demand for services. To develop and compare regional and national 
measures of unmet (or deferred) demand for adult education services and excess (or under 
utilized) service capacity, and to assess tl^e extent to which improved management of existing 
adult education resources might bring supply and demand into closer balance. 


Rpt 1: ch 4 
Rpt 4: ch 5 


6. Participation rates of target populations. To develop estimates of the size and composition 
of target populations for each of the program's service components and, by relating these 
estimates to data on program clients, to assess levels and rates of program participation for 
these target populations. 


Rpt 4: ch 5 


7. Learning gains. To develop estimates of average learning gains as related to hours of 
instruction and/or tutoring for each program component and, by applying these estimates to 
data on participation, to assess aggregate learning outcomes generated by the program over a 
one-year period. 


Rpt 4: ch 3, 
ch 4 & ch 5 


8. Service costs. To develop estimates of average service costs as related to hours of 
instruction and/ or tutoring for each program component and, by relating these estimates to 
data on participation and leanung gains, to assess the service costs associated with producing 
successful outcomes. 


Rpt 1: ch 4 
Rpt 4: ch 2 
Cost Study 
Rpt 


9. Employment outcomes. To evaluate the extent to which sustained program participation is 
significantly associated with favorable employment outcomes, using employment outcomes of 
early leavers as the standard of comparison. 


Rpt 4: ch 4 


10. Dissemination. To stimulate wider interest in a discussion of policy issues in adult 
education by means of timely dissemination of findings and interim reports, commissioned 
papers on selected issues, and a national conference at the conclusion of the study. 


Bulletins & 
Interim Rpts 
to the field 


11. Independent research. To facilitate independent research on adult education by issuing 
unit-record data files for the national samples of service providers and new clients, along with 
provisions for linking these two files and high-quality user-oriented technical documentation. 


Data tapes 
& 

documenta- 
tion 


12. Analytic agenda. To develop recommendations concerning future analytic agendas for 
adult education, with special reference to further uses of data from the 1992 National Survey of 
Adult Literacy and the 1990 Census. 


Exec. 

Summary 

Rpt 



A-3 



■5 - 

ERIC 



APPENDIX B 

Data Collection Form: Telephone Follow-up Survey Form 



Note: 

Copies of other data collection fomis are included as Appendices to prior 
reports, as follows: 

First Interim Report - Profiles of Service Providers 
Universe Survey of Adult Education Providers 
Comprehensive Program Profile Survey 

Second Interim Report - Profiles of Client Characteristics 
Client Intake Record: Part A and 
Client Intake Record: Part B 

Third Interim Report - Patterns and Predictors of Client 

Attendance 

Client Update Record 
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Telephone Follow-up Survey Form 
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TELEPHONE FOLLOWUP SURVEY FORM 



Q#l I understand that you were in a (ABE/ESL/ASE/GED) program at (LOCATION) 
about six months ago. Is that correct? 

Yes n No(no one in household 

participated in this program)IF 
THE PARTICIPANT ANSWERED 
NO TO THIS QUESTION, THEN 
THE INTERVIEW WAS 
TERMINATED. 

Q#2 Did you attend THREE OR MORE classes of the (ABE/ESL/ASE/GED) program? 

□ Yes-3 OR MORE CLASSES ATTENDED □ No-2, 1, CLASSES ATTENDED 

□ No - NO CLASSES ATTENDED □ [DK/Refused] 



Note: Except for Q#19, the remaining questions were asked only of clients 
with 3 or more hours of instruction. 



Q#3 Did you learn what you wanted to learn from the instruction that you received? 
□ Yes □ No □ DK/Refused 



To what extent did the educational class or training program help you? 

(Questions Q#4 through Q#17 were asked only of clients who had indicated when they 
enrolled in the program - on Intake Record Part B - that this was a "very important" 
reason that they were taking adult education instruction. Those who indicated the 
reason was "somewhat important" or "not important" are designated NA.) 

Q#4 To what extent did the educational class or training programs help you to 
improve reading/writing skills? 

□ A Lot ^ Somewhat 

□ Not At AU □ [DK/Rehised] 

Q#5 To what extent did the educational class or training programs help you to 
improve math skills? 

□ A Lot ^ Somewhat 

□ Not At All □ [DK/Rehised] 



Telephone FoUowup Survey Page 2 

Q#6 To what extent did the educational class or training programs help you to 
improve speaking and listening skills? 

□ A Lot D Somewhat 

□ Not At All □ [DK/Refused] 

Q#7 To whit extent did the educational class or training programs help you to enter 
a vocational or job training program? 

□ A Lot n Somewhat 

□ Not At All □ [DK/Refused] 

Q#8 To what extent did the educational class or training programs Help you to get a 
job or a better job or qualify for higher pay? 

n A Lot O Somewhat 

□ Not At All □ [DK/Refused] 

Q#9 To what extent did the educational class or training programs help you to 
improve job performance? 

n A Lot D Somewhat 

□ Not At All □ [DK/Refused] 

Q#10 To what extent did the educational class or training programs help you to 
qualify for United States citizenship? 

□ A Lot CD Somewhat 

□ Not At All □ [DK/Refused] 

Q#ll To what extent did the educational class or training programs help you to feel 
better about yourself? 

□ A Lot n Somewhat 

□ Not At All □ [DK/Refused] 

Q#12 To what extent did the educational class or training programs help you to 
contribute better to your family and community? 

□ A Lot D Somewhat 

□ Not At All □ [DK/Refused] 
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Telephone FoUowup Survey Page 3 

Q#13 To what extent did the educational class or training programs help you to help 
your children with schoolwork? 

□ A Lot ^ Somewhat 

□ Not At All □ [DK/Refused] 

Q#14 To what extent did the educational class or training programs help you to 
become less dependent on others for help? 

□ A Lot Somewhat 

□ NotAtAU □ [DK/Refused] 

Q#15 To what extent did the educational class or training programs help you to make 
others feel better about you? 

□ A Lot ^ Somewhat 

□ NotAtAU □ [DK/Refused] 

Q#16 To what extent did the educational class or training programs help you to enter 
college? 

□ A Lot O Somewhat 

□ NotAtAU □ [DK/Refused] 

Q#17 Did you attend the educational class or training program to get a GED? 

□ Yes □ No □ DK/Refused 

Q#18 Did you attend the educational class to earn a regular High School Diploma? 

□ Yes □ No □ DK/Refused 



Telephone Followup Survey Page 4 



Following 0^19 respondents were asked: "Was that all?" Up to three other 
responses were recorded. 



Q#19 What was your major reason for leaving your class or instructional program? 

D Client Left Satisfied - Completed Program 

n Client Left Satisfied - Completed Required Attendance 

□ Client Left Satisfied - Got What Went For/Achieved Personal Goals 
D Client Left Satisfied - OTHER (eg- enrolled in school, got job) 

n Left/Outside Events-f- Personal Illness, Health Problems 

0 Left /Outside Events + Family Responsibilities (includes Child Care) 

n Left /Outside Events + Transportation Problems 

D Left /Outside Events -f- Change of Job or Work Responsibilities 

□ Left/Outside Events + OTHER reasons not to do with the course itself 

□ Left Due To Courses - Too many Forms/To many Tests to Take 

D Left Due To Courses - Personal Embarrassment /Personal Discomfort 

n Left Ehie To Courses - Lack of Progress /Dissatisfied with Program 

□ Left Due To Courses - Took too much Time & Energy (eg- too far to drive) 

□ Left Due To Courses - Interfered witli other (preferred) activities 

□ Left Due To Courses - Worried about Personal Safety (eg- bad neighborhoods) 
D Left Due To Courses - OTHER reasons reflective of the program 

□ NO REASONS GIVEN+ (can't say, just left) 

□ -NO MORE APPLY- 



Questions Q#20 through Q#28 were asked of those clients who had a job when 
tliey enrolled. 



Q#20 I see that you were working when you enrolled in the program. 

Are you working now for pay? 

□ Yes (if yes, go to Q#21) □ No af no, go to Q#25) □ DK/Refused 
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Telephone Followup Survey Page 5 

Q#21 Do you still have that same job you had when you enrolled in the program? 

□ Yes (if yes, go to Q#22) □ No (if no, go to Q#26) □ DK/ Refused 

Q#22 Did what you learned in the program help you with that job? (i.e. the same job 
you had when you enrolled) 

□ Yes □ No □ DK/Refused 

Q#23 Are you working at more than one job? (i.e., at a job in addition to the one you 
had when you enrolled) 

D' Yes (If yes, go to Q#24} D No (if no, go to Q#52) □ DK/Refused 

Q#24 Did what you learned in the program help you get that second job? 

□ Yes (if yes, go to Q#32) □ No (If no, go to Q#32) □ DK/Refused 

Q#25 Have you worked regularly at any time since you left the program? (asked of 
those not working now,) 

n Yes (if yes go to Q#28) d No (if no, go to Q#32) □ DK/Refused 

Q#26 Did what you learned help you get a better job than you had? (asked of those 
who changed jobs.) 

□ Yes □ No □ DK/Refused 

Q#27 Did what you leamed in the program help you in your current job? (asked of 
those who changed jobs.) 

□ Yes □ No □ DK/Refused 

Q#28 Did what you leamed in the program help you with that job? (asked of those 
not currently working, but who worked regularly since leaving the program.) 

□ Yes □ No □ DK/Refused 



Questions Q#29 through Q#31 were asked of those clients who were not 
working when they enrolled. 



Q#29 I see that you were not working when you enrolled in the program. Are you 
working now for pay? 

□ Yes (if yes, go to 31) □ No (if no, go to Q#3o) □ DK/Refused 

Q#30 Have you worked at any time since you left the program? 

□ Yes (if yes, go to 31) □ No (if no, go to Q#32) CH DK/Refused 



i. 1 
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Q#31 Did what you learned in the program help you get the job? 
□ Yes □ No □ 



DK/Refused 



Q#32 Do you have children under the age of 6? 



□ 



Yes 



□ 



No 



□ DK/Refused 



Q#33 How often do you read to or with your children? 

Almost Never 
n About Once Or Twice A Month 

□ About Once A Week 

□ Nearly Every Day 

□ DK/Refused 



Q#34 Do you feel you need or would like additional educational classes or training? 
□ Yes . □ No □ DK/Refused 
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Following Q#35 respondents were asked: "Is that alP" Up to ei^ht 
responses were recorded. waiaii. up to eight 



other 



Q#35 What is your reasoit for waitting additional classes or training^ 

i-J Academic. . . Reading/ Writing 
^ Academic. . . Mathematics 



D Academic. . . Understanding EngUsh 

□ Credentials . To Obtain GED/High School Diploma 

□ Credentials . Enter Vocational Or Job Training Program 
n Credentials . Enter College 

D Employment. . Get A Job Or Better Job 

□ Employment. . Improve Job Performance 
D Employment. . Increase Income 

□ Employment. . Satisfy Employer Or Program Requirements 
LJ Personal. . . Prepare For Citizenship 

D Personal. . . Improve Self/Feel Better About Myself 
n Personal. . . Contribute To Children 

□ Personal. . . Contribute To Church, Community 

□ Personal. . . Become Less Reliant On Others 
O Other Reason 

n DK/Refused 

n -NO MORE APPLY- 

For the next few questions, describe how you usually felt about thp 
instruction you received in your adult education prog^^^^^^^ 

Q#36 Were the lessons at a level you could understand. 

□ Always (1) □ Most Of Tl^e Time (2) 

n Sometimes (3) □ Never (4) 

D [Not Applicable/Do Not Remember/Refused] 
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0#37 Were the mstructional materials helpful... 

r-i A, „c □ Most Of The Time 

□ Always 

D Sometimes ^ Never 

□ [Not Applicable/Do Not Remember /Refused] 

0#38 Did your teacher work with small groups of students... 

rlT D Most Of The Time 

□ Always 

□ Sometimes ° Never 

D [Not Applicable/Do Not Remember /Refused] 

Q#39 Was the teacher or aide available to help you individually with the lessons... 

□ Always ° Most Of The Time 
O Sometimes D Never 

□ [Not Applicable/Do Not Remember /Refused] 

O#40 Was there enough practice time in the class... 

D Most Of The Time 

□ Always '-^ 

□ Sometimes ° Never 

□ [Not AppUcable/Do Not Remember /Refused] 

0#41 Were questions from students encouraged by the teacher... 

^ □ Most Of The Time 

□ Always 

□ Sometimes ° Never 

□ [Not Applicable/Do Not Remember /Refused] 

nM2 Were your questions answered... 

, ' □ Most Of The Time 

□ Always 

i-i ^ ^ D Never 

□ Sometimes ^ 

□ [Not Applicable/Do Not Remember /Refused] 

*^ .r.tr.«uters in vour instructional program? 
Q#43 Did you have access to computers m you DK/Refused 

n X/ □ No 

□ Yes 
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Q#44 Were they helpful to you— 

□ Always □ Most Of The Time 

□ Sometimes d Never 

□ [Not Applicable/Do Not Remember/Refused] 

QMS Are you attending any educational classes or training program now? 

□ Yes □ No □ DK/Refused 



Following Q#46 respondents were asked: "Is that all?" Up to foiir other 
responses were recorded. 



Q#46 What kind of educational class or program are you attending? 

n English Language Skills 

□ Math Skills 

□ GED/High School 

n Vocational/Job Training 

D Corrmiimity College /College Level 

n Citizenship 

□ Other 

□ DK/Refused 

□ -NO MORE APPLY- 

Q#47 Do you plan to enroll in any educational classes or training program in the 
future? 

□ Yes □ No □ DK/Refused 

QMS When do you plan to take the classes/program? 

□ Before Next Year 

□ Next Year 

□ Within The Next Three Years 

□ Longer Than Three Years From Now 

□ DK/Refused 
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FoUowing Q#49 respondents were asked: "Is tiKat all?" Up to five other 
responses were recorded. 



Q#49 What educational classes or training program do you plan to take? 

n English Language Skills 

□ Math SkiUs 

□ GED/High School 

n Vocational/Job Training 

0 Community College/College Level 

D Citizenship 

□ Other 

□ DK/Reifused 

□ NO MORE APPLY- 
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APPENDIX C 

Sampling Design, Data Collection, and Standard Errors 




Sample of Programs and Clients 

The study objectives called for collecting of infoimation for a probability 
sample of approximately 50,000 new clients in 150 programs. To reach these 
objectives, tike study attempted to develop a sample of clients with a sampling rate of 
approximately 1 in 60 new clients. 

The first step in the selection process was to group the programs by title four 
Census regions. Within each region, they were then ordered by their enrollments for 
the previous year as reported in their response to the Universe Survey of Adult 
Education Providers. Eighteen programs were determined to be so large as to fall 
into the study witiK certainty (a probability of selection equal to one). These 18 
"certainty programs" all had enrollments of 20,000 or more, or more than 1/150 of all 
projected new enrollments. 

To ensure titieir inclusion, small programs witiK 300 or fewer clients were also 
treated separately. Every sixtieth small program was selected for inclusion in the 
Study. There were 20 small programs selected. 

Since the study target was a total of approximately 150 programs and 50,000 
clients, we determined the total erirollment of all programs that were neither certaint}'^ 
programs nor small programs and divided by 112, which was the number of 
programs that were needed to reach the target of 150. The result-21,948 clients-was 
the sampling interval that was used to select the remaining "mid-sized" programs. 
Within each Census region, the mid-sized programs were ordered by size and sample 
programs were selected systematically with probability proportionate to their 
reported enrollment, using a random starting point between 1 and 21,948. 

This process provided the study with the programs to be selected. In tihe few 
cases (25) where programs were unable to participate, they were replaced by 
randomly selecting one of the programs that fell closest to the originally selected 
program in the list of providers. 

In order to maintain the desired overall selection probability for clients at the 
level of about 1 in 60, it was necessary to select a sample of sites and, in some 
instances a sample of clients within sites, for each of the certainty programs and most 
of the mid-sized programs selected for the sample. In order to make it relatively 
easy for programs to participate in the study, we adopted a strategy that limited the 
number of their sites included in the study sample but adequately reflected variations 
in size and other characteristics of interest. For the small sites, all sites and clients 
are included. For large and mid-sized programs the process followed the following 
guidelines: 
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• Lists of program sites and their enrollments were obtained. 

• When a program had several sites they were assigned to one of a 
number of "clusters." The number of clusters developed for a program 
depended on the enrollments which they reported and number of 
sample clients needed per program. 

• If the number of clients in a site or cluster was significantly larger than 
the number needed for the sample from that program, a sample of 
clients in that site or cluster was selected. 

In two instances, programs that were selected from the group of small 
programs closed after their selection for participation. These programs were not 
replaced, inasmuch as the active "life" of programs is of interest to the study. Ten 
other programs, subsequent to being selected and having agreed to participate in the 
Study, elected to not participate. These programs, none of which were certainty 
programs, left the Study at too late a date to be replaced. 

Data Collection Procedures and Processing 

The research design for the evaluation incorporated the following data 
collection activities: 

Universe Survey: This was a survey of all providers of adult education which 
received monies through the Basic State Grants portion of the federal Adult 
Education Act during 1989-1990. A mail survey, with extensive follow-up, was 
implemented in October and November 1990. Responses were obtained from 2,619 
(93 percent) of the 2,819 local service providers. Of the respondents, 306 (12 percent) 
were interviewed by telephone, and were asked only a subset of the questions from 
the mail questionnaire. 

Comprehensive Program Profile: This was a survey to be completed by the directors of 
the programs participating in the longitudinal phase of the study. Of the 150 selected 
programs, 131 actually completed profiles and began participation in the study. 

Client Intake Record: This information was obtained on the sample of clients who 
enrolled in adult education programs during the twelve month period which began 
in mid-April 1991. The intake record consisted of two parts. The first part (Intake A) 
was completed from program intake records. The second part (Intake B) was to be 
completed by or for newly enrolled clients who attended at least one instructional 
session. We received Intake A forms on 22,548 clients and Intake B forms on 13,845 
clients (82 percent, 90 percent weighted, of those clients with Update forms indicating 
they began instruction). These forms provided demographic information about 
clients and their reasons for participation in the program. Exhibit C.l compares 



characteristics of clients for whom we received an Intake B with clients for whom we 
did not receive an Intake B, after weight adjustments have been made (the 
characteristics reported are from Intake A forms). 

Exhibit C.1 

Comparison of Selected Characteristics of Clients With and Without Client Intake 
Record Part B Who Received At Least 1 Hours of Instruction 







Clients With B Forms 


Clients Without B Forms 


Variable 


Value 


ABE 


ASE 


ESL 


ABE 


ASE 


ESL 




Male 
Female 


40% 
60 


38% 
62 


46% 
54 


40% 
60 


37 % 
63 


50 % 
50 




American Indian or 
Alaskan Native 


3 


5 


0 


1 


2 


1 




Asian or Pacific 
Islander 


2 


2 


20 


3 


3 


28 


Race 
Ethnicity 


Black, not of Hispanic 
Origin 


26 


14 


1 


23 


20 


2 




Hispanic 


10 


11 


72 


9 


9 


59 




White, not of Hispanic 
origin 


59 


68 


6 


64 


66 


10 




16-21 


32 


41 


22 


32 


43 


21 


Age 




30 


29 


40 


29 


25 


37 


31-44 


28 


23 


27 


30 • 


23 


32 




45 & Older 


in 


7 


11 


9 


8 


10 




I^one 


83 


86 


46 


85 


91 


57 




GED Certificate 


3 


3 


1 


3 


2 


0 


Highest 
Diploma 


High School Diploma 
Technical Certificate 
A.A. Degree 


11 
1 
0 


7 
1 
0 


30 
9 
3 


12 
0 
0 


5 
0 
0 


25 
4 
2 




4 yr College Graduate 


0 


0 


8 


0 


0 


6 




Other 


1 


1 


3 


0 


1 


6 


Marital 
status 


Now Married 

Widowed 

Divorced 


31 
2 

1 


31 
1 

10 


38 
2 
3 


43 
1 

10 


33 
3 
9 


52 
1 
3 



1 X 



Variable 


Value 

Never married 


Clients With B Forms 


Clients Without B Forms 


ABE 

7 
50 


ASE 

6 

52 


ESL 

4 

53 


ABE 

4 
42 


ASE 

3 
53 


ESL 

2 
43 




North Central 


31 


29 


6 


38 


36 


18 




Northeast 


22 


16 


5 


11 


9 


8 


Region 
















South 


37 


34 


6 


43 


35 


14 




West 


10 


22 


83 


8 


20 


60 



Client Update Record: This form was completed by program staff and provided 
instructional and attendance data on 18,461 of the sampled clients for 18 months after 
they began instruction. 

Client Test Data: When possible programs provided scores on standardized tests of 
basic skills given at the time of enrollment for clients in the study. One or more 
posttests were administered with the goal being that tests be given after 70 and 140 
hours of instruction. Posttest data were received on 1,919 clients from 65 local 
programs. 

Telephone Follow-up Interview: Questions regarding employment status, 
accomplishment of clients' personal objectives, and an assessment of the instruction 
received were asked of all clients who had completed Intake B and who had been 
separated from the program for a period of six months by the end of data collection 
(October 1993) through a telephone survey. Telephone surveys were attempted for 
10,273 clients and completed for 5,401. 

To keep costs within reasonable bounds, the evaluation was designed to rely 
heavily on staff from local programs for the compilation and transmission of data. 
Local persormel were trained in the data collection requirements in the Spring of 
1991. Monitoring and related quality control procedures were implemented on an 
on-going basis, and programs regularly provided follow-up information by telephone 
and mail. Where necessary, supplemental training was provided. 

All of the data collection instruments were designed so they could fimction as 
source documents for data processing purposes. 'iAHien data collection instruments 
were received, they were carefully reviewed for completeness and legibility by 
program staff. Where needed, follow-up telephone calls to local programs were 
made to clarify or complete particular items. Following this manual edit, coding of 
open-ended responses was done in accordance with standard research procedures. 
Forms were then optically scanned, and computer editing was conducted under the 



program staff. Where needed, follow-up telephone calls to local programs were 
made to clarify or complete particular items. Following this manual edit, coding of 
open-ended responses was done in accordance with standard research procedures. 
Forms were then optically scanned, and computer editing was conducted under the 
guidance of specific editing instructions developed for each form. These generally 
consisted of checks for completeness, accuracy, internal consistency, and out-of-range 
values. When needed, local programs were contacted again to obtain missing 
information or resolve anomalies in the data. 

Sampling Weights 

The weights used to develop estimates from the data collected in the national 
evaluation were designed to reflect the selection probabilities of programs and clients, 
and to minimize any biases from the exclusion of sample programs that did not agree 
to participate in the study, either initially, or at the intake stage. Nonresponse 
adjustments were developed separately for each Censiis region and each of the three 
program size groups: large, intermediate, and small. The sampling weights and 
nonresponse adjustments used for estimates of program characteristics and estimates 
based on intake data are described in detail in appendix A of the First Interim Report 
and appendix C of the Second Interim Report. 

Because two of the programs that failed to provide adequate client update data 
were very large certainty programs (Miami, Florida and Chicago, Illinois), we opted 
not to make further nonresponse adjustments for estimates based on update data. 
Such adjustments would have substantially increased the sampling errors for these 
estimates and it was not clear that they would have been effective in controlling 
noruresponse bias. Thus, the estimates based on client update information are 
estimates of national totals, ratios, and percents for all programs other than the two 
that were excluded. An analysis of intake estimates with and without the inclusion 
of these two programs did not show any substantial differences in estimates of ratios 
and percents. 
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The Telephone Survey 



The Telephone Follow-up Survey provides information about the quality of 
instruction, the reasons for termination, and the results of instruction from a 
subsample of clients 6 months after they left the program. The survey was of clients 
who had completed at least one hour of instruction and for whom we received a 
completed Client Intake Record: Part B (the data on Part B includes the client's 
telephone number and other information which permits a telephone follow-up). 
Attempts were made to contact all clients who met the selection criteria. 

An examination of the sample of clients included in the Follow-up Telephone 
Survey revealed that one could predict with considerable accuracy whether a client 
met the telephone survey selection criteria (i.e., whether an interview would be 
attempted), and, given a client's selection, whether contact would be made\ The 
high degree of predictability is indicative of a nonrandom system, and indicates that 
tabulation of responses to the telephone survey will yield biased estimates of 
responses that would be given by the entire population. Hence, for this report, a 
separate weighting variable was calculated and used for the telephone sample. 
Briefly stated, the weights were adjusted by using a probit regression to predict the 
probability of a completed telephone interview and using that probability as an 
adjustment to the original client-level weight. The adjustment was made by 
subtracting the probability obtained from the probit from 1 and dividing the orginal 
client weight by the result. Hie result is the adjusted weight for the telephone survey 
respondents. 

Exhibit C.2 shows how the respondents to the telephone survey compare to the 
population of all those receiving instruction. The weighted groups are very similar, 
with the completed sample having fewer ESL clients and more ASE clients than are 
seen in the population, and the telephone sample accumulating considerably fewer 
hours and weeks of instruction. In order to be eligible for the telephone follow-up 
one had to be inactive for six months, so clients who attend for long periods were 
unlikely to be included, thereby reducing the median hours and weeks of instruction. 



^ The regression used a total of 33 variables (mostly dichotomous variables derived from intake 
form A arid update data). The following variables were significant at .01: placement in ESL or ABE, 
female (negative), American Indian, Asian, Black, H?^'^anic, unknown cotmtry of birth, home 
ownership (negative), married (negative), and weeks enrolled in year 1 (negative). The measure of 
model fit, -2 Loglikelihood (-2LLR)= 1292 p = .0001. -2LLR yields a measure of statistical significance 
of the improvement over the null model (see, for example Henri Theil 1971., Principles of 
Econometrics, John Wiley and Sons, New York.) 
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Exhibit C.2 

Comparison of Selected Characteristics of Weighted Samples of Telephone Survey 
Respondents and All Clients With 1 or More Hours of Instruction 



Variable 




Population 


Telephone 


Value 


ABE ASE ESL 
(24%) (30%) (46%) 


ABE ASE ESL 
(24%) (37%) (39%) 


Gender 


Male 
Female 


40 38 46 
60 62 54 


43 39 48 
57 61 52 


Race 
Ethnicity 


American Indian or 
Alaskan Native 

Asian or Pacific 
Islander 

Black, not of Hispanic 
Origin 

Hispanic 

White, not of Hispanic 
origin 


R n 

O D \J 

2 2 21 

25 15 1 

10 11 71 
60 68 6 


2 2 20 

23 13 2 

9 9 71 
61 70 6 


Highest 
Diploma 


None 

GED Certificate 

High School Diploma 

Technical Certificate 

A.A. Degree 

4 yr College Graduate 

Other 


89 97 47 
3 3 1 

11 7 30 
1 1 9 
0 0 3 

0 0 8 

1 1 3 


82 85 48 
3 3 1 

13 8 28 
12 7 
0 0 2 

0 19 

1 1 4 


Marital 
status 


Now Married 
Widowed 
Divorced 
Separated 
Never married 


32 31 39 
2 2 2 

10 10 3 
6 5 4 

49 52 53 


30 31 43 
2 1 1 

10 10 3 
6 6 3 

52 52 49 
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Variable 




Population 


Telephone 


Value 


ABE ASE ESL 
(24%) (30%) (46%) 


ABE ASE ESL 
(24%) (37%) (39%) 


Reason for 
Enrollment 


Required by employer 

Required by another 
program or agency 

To satisfy family or 
friends 

To satisfy other 
personal or 
employment goals 


2 2 5 
13 9 4 

8 9 6 

77 80 84 


3 2 6 
11 9 5 

9 9 7 

78 80 82 


Bom in U.S. 


Yes 
No 


91 92 2 
9 8 98 


92 92 4 
8 8 96 


V^UlTcill 

living 

arrangement 


Own home 
Renting 

Temporarily living 
with someone else 

No regular place of 
residence 


24 28 8 
48 43 70 
27 28 ■ 21 

111 


24 26 9 
47 42 70 
28 30 20 

2 11 


How often 
do you read 
to your 
kids? 


Almost never 

About once or twice a 
month 

About once a week 
Nearly everv day 


26 22 39 
11 14 10 

28 28 23 
34 36 28 


26 24 34 
11 13 13 

27 27 24 
36 37 30 


Live at the 
same 

address as 5 
years ago 


Yes 


34 35 9 
66 65 91 


37 37 12 
63 63 88 


Did you live 
outside the 
U.S. 


Yes 
No 


9 6 77 
91 94 23 


8 6 73 
92 94 27 


Did you live 
in the same 
state 


Yes 
No 


81 79 72 
19 21 28 


84 80 75 
16 20 25 


Did you live 
in the same 
county 


Yes 
No 


86 82 87 
14 18 13 


85 83 87 
15 17 13 
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Variable 


Value 


Population 


Telephone 


ABE 


ASE 


ESL 


ABE 


ASE 


ESL 






(24%) 


(30%) 


(46%) 


(24%) 


(37%) 


(39%) 




North Central 


32 


30 


7 


32 


30 


8 




Northeast 


21 


15 


5 


24 


18 


7 


Region 


South 


38 


34 


6 


36 


32 


8 




West 


9 


22 


82 


9 


20 


76 


Hours of 


Median 


36 


28 


114 


34 


29 


55 


instruction 


Mean 


84 


68 


224 


61 


53 


93 


Weeks of 


Median 


11 


8 


21 


10 


8 


10 


instruction 


Mean 


20 


17 


30 


16 


14 


17 
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Sampling Errors 



The client statistics presented in this report are based on a stratified, multistage 
probability sample as described above. All program and client estimates except those 
based on the Universe Survey of programs are subject to sampling error. For 
estimates based on data from the Comprehensive Program Profile, formulas for direct 
estimation of sampling errors of program characteristics were presented in appendix 
D of the First Interim Report. ^ Estimates of ratios and percents based on the sample 
of 131 programs that completed profiles have relative large sampling errors: the 
primary design goal of the program sample design was to provide tihe basis for an 
efficient sample of clients, not to provide estimates of program characteristics. 

Estimated standard errors are presented for selected program characteristics in 
exhibit C.3 and for selected client characteristics in exhibit C.4. Standard errors were 
estimated using PC CARP software developed by the Statistical Laboratory at Iowa 
State University. The standard errors can be used to construct confidence intervals 
for estimates based on probability samples. For example, a 95 percent confidence 
interval for the estimated percent of ESL clients in the age group 31-45 would be 29.1 
± (1.96 X 1.2). Thus, 95 percent confidence interval is between 26.7 percent and 31.4 
percent. A 95 percent confidence interval for the percent of ABE clients receiving 12 
or more hours of instruction would be 67.2 ± (1.96 x 2.0) or between 63,3 percent and 
71.1 percent. 
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Exhibit C.3 

Standard Errors for Estimated Percentages of Selected Variables 



Program Characteristic 


Percent of programs 


Standard error 


Sponsor 






Local public school 


70 % 


5.7 % 


Commimity college 


15 


3.9 


PLO 


9 


3.9 


Technical instihite 


5 


2.5 


Regional education service agency 


1 


0.1 


Urbanicity 






Large city in major metropolitan area 


3 % 


2.8 % 


Remainder of major metropolitan area 


9 


2.5 


Smzdl metropolitan area 


23 


4.7 


Norunetropolitan area 


64 


4.8 


Percent full-time staff 


13 % 


1.3 % 


Clients' Support Service Needs 


Estimate of Percent of 
Programs Meeting Needs 
of Clients 


Standard 
Error 


Counseling 


91 % 


3.9 % 


Job search assistance 


82 


4.9 


Transportation 


52 


7.0 


Child care 


43 


6.6 


Translator services 


39 


6.6 


Financial assistance 


37 


6.3 


Case management 


31 


6.1 


Health services 


30 


6.4 
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Exhibit C.4 

Standard Errors for Estimated Percentages from Selected Data Collected from 

Clients and Program Directors 



Characteristic 




ESL 


ABE 




ASE 




Estamate 


Standard 
error 


Estimate 


Standard 
error 


Estimate 


Standard 
error 


Region 














Northeast 


20.6 % 


3.7 % 


36.9 % 


2.8 % 


42.4 % 


4.5 % 


North Central 


24.4 


1.6 


35.2 


2.6 


40.4 


2.5 


South 


19.9 


5.8 


39.2 


5.8 


40.9 


6.5 


West 


81.9 


10.6 


4.3 


2.5 


13.8 


82 


Race/ 
Ethnicity 














American Indian/ 
Alaskan Native 


0.4 


0.2 


1.8 


0.9 


3.9 


1.9 


Asian or Pacific 
Islander 


18.9 


11 


1.7 


0.5 


1.8 


0.8 


Black, not of 
Hispanic Origin 


4.3 


2.1 


38.3 


9.0 


19.7 


4.8 


Hispanic 


68.5 


4.1 


11.6 


3.0 


12.8 


4.0 


White, not of 
Hispanic Origin 


1 Q 

/.y 


1 Q 

i.y 


AC,. C 

4o.o 




ol.o 


o.U 


hex 














Female 


4D.Z 


1 7 
1./ 


AO Q 


1 7 
i ./ 


/in 

4U.0 




Nlale 


DO,/ 


1 7 
1./ 


^7 1 


1 7 
i./ 






Age 














16-21 


21.9 


0.7 


27.9 


1.8 


44.4 


2.0 


22-30 


38.8 


0.7 


29.8 


0.9 


27.3 


1.0 


31-45 


29.1 


1.2 


31.8 


1.7 


22.5 


1.0 


Over 45 


10.1 


0.9 


10.6 


1.6 


5.8 


0.9 


Began Instruction 














Yes 


92.6 


2.0 


86.0 


2.0 


85 


2.0 


Received 12 
hours of 
instruction 














Yes 


84.8 


4.0 


67.2 


2.0 


63.7 


1.0 
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Exhibit D*l 

Proportion of All New Beginning Clients Exiting in Specified 
Number of Months from Time of Entry 



Niimber of 

Months 
Before Exit 


Time of Entry 
(Weighted niomber of enrolling clients] 


1 


Apr-May 
(134,002) 


jion-jul 
(129,080) 


Aug-Sep 
(502,763) 


vJCt-lNOV 

(195,324) 


Dec-Jan 
(214,802) 


reD-iviax 
(298,444) 


lOtol 

(1,474,415) 


U-l 


0.42 


0.45 


0.24 


0.37 


0.34 


0.33 


0.33 


1 9 


0.09 


0.08 


0.13 


0.11 


0.11 


0.11 


0.11 




0.06 


0.07 


0.10 


0.11 


0.12 


0.11 


0.10 




0.03 


0.05 


0.03 


0.06 


0.09 


0.06 


0.05 




0.06 


0.04 


0.05 


0.05 


0.05 


0.03 


0.04 




0.03 


0.03 


0.05 


0.06 


0.04 


0.03 


0.04 


6-7 


0.02 


0.02 


0.04 


0.04 


0.01 


0.02 


0.03 


7-8 


0.02 


0.03 


0.08 


0.02 


0.02 


0.02 


0.04 


8-9 


0.02 


0.01 


0.04 


0.02 


0.02 


0.02 


0.03 


9-10 


0.02 


0.04 


0.02 


0.02 


0.02 


0.02 


0.02 


10-11 


0.02 


0.04 


0.01 


0.01 


0.02 


0.01 


0.01 


11-12 


0.06 


0.01 


0.02 


0.02 


0.01 


0.01 


0.02 


Continuing 


0.15 


0.14 


0.20 


0.13 


0.15 


0.24 


0.18 
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Exhibit D.2 

Proportion of All New Beginning ESL Clients Exiting in Specified 
Number of Months from Time of Entry 



Number of 

Months 
Before Exit 


Time of Entry 
(Weighted number of eruroUing clients) 


Apr-May 


Jun-Jul 


Aug-Sep 


Oct-Nov 


Dec-Jan 
yyOfOD/ ) 


Feb-Mar 


Total 


0-1 


0.31 


0.35 


0.18 


0.31 


0.28 


0.25 


0.24 


1-2 


0.07 


0.05 


0.14 


0.09 


0.08 


0.08 


0.10 


2-3 


0.05 


0.07 


0.09 


0.11 


0.09 


0.09 


0.09 


3-4 


0.03 


0.07 


0.03 


0.05 


0.07 


0.07 


0.05 


4-5 


0.08 


0.04 


0.04 


0.04 


0.05 


0.02 


0.04 


5-6 


0 0*^ 




0 OS 






0 OS 


0 OS 


6-7 


0.02 


0.02 


0.03 


0.04 


0.01 


0.01 


0.02 


7-8 


0.02 


0.02 


0.09 


0.03 


0.02 


0.02 


0.05 


8-9 


0.01 


0.01 


0.04 


0.03 


0.03 


0.03 


0.03 


9-10 


0.02 


0.06 


0.02 


0.03 


0.03 


0.02 


0.03 


10-11 


0.01 


0.07 


0.01 


0.01 


0.03 


0.00 


0.02 


11-12 


0.12 


0.00 


0.02 


0.02 


0.02 


0.01 


0.03 


Continumg 


0.23 


0.20 


0.25 


0.18 


0.23 


0.35 


0.26 
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Exhibit D.3 

Proportion of All New Beginning ABE Clients Exiting in Specified 
Number of Months from Time of Entry 



Number of 

Months 
Before Exit 


Time of Entry 
(Weighted number of enrolling clients) 


Apr-May 


Jun-Jul 


Aug-Sep 

J. fX.>J\J f 


Oct-Nov 
f 55 374^ 


Dec-Jan 
f47 7881 


Feb-Mar 
(40-9181 


Total 
(285^76) 


0-1 


0.49 


0.42 


U.oU 




u.^ 


U.*i*t 


n X7 

\j,o/ 


1-2 


0.10 


0.10 


\J.lO 


n 10 

U.iZ 


n 19 


n 1 1> 


0 19 


2-3 


0.06 


c\ no 






n iA 


nil 


nil 


3-4 


0.04 


0.04 


U.Uo 


U.Uo 


nil 






4-5 


0.06 


0.04 


U.Uo 


U.UO 


U.Uo 


U.UrJb 




5-6 


0.03 


0.05 


0.06 


0.06 


0.02 


0.02 


0.04 


6-7 


0.03 


0.02 


0.05 


0.04 


0.01 


0.01 


0.03 


7-8 


0.01 


0.04 


0.05 


0.01 


0.01 


0.03 


0.03 


8-9 


0.04 


0.01 


0.04 


0.02 


0.03 


0.01 


0.03 


9-10 


0.02 


0.02 


0.01 


0.02 


0.02 


0.02 


0.01 


10-11 


0.02 


0.02 


0.00 


0.01 


0.01 


0.01 


0.01 


11-12 


0.01 


0.02 


0.01 


0.01 


0.01 


0.01 


0.01 


Continuing 


0.10 


0.1.3 


0.15 


0.09 


0.09 


0.10 


0.11 
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Exhibit D.4 

Proportion of All New Beginning ASE Clients Exiting in Specified 
Number of Months from Time of Entry 



Number of 

Months 
Before Exit 


Time of Entry 
(Weighted niimber of enrolling clients) 


Apr-May 
(i^ 095 ^ 


Jun-Jul 
(A.7 704 ^ 


Aug-Sep 


Oct-Nov 


Dec-Jan 
f73 1 36^ 


Feb-Mar 


Total 


0-1 


0.55 


0.59 


0.35 


0.43 


A vl 1 

0.41 


0.46 


0.44 


1-2 


0.12 


O.Oo 


0.13 


A 1 n 

0.10 


A 1 >l 

0.14 


A 1 yl 

0.14 


A % > 

0.12 


2-3 


0.08 


0.07 


nil 

0.11 


A AA 

0.09 


A 1 /I 

0.14 


A -I C 

0.15 


All 

0.11 


3-4 


0.02 


0.04 


A Ail 

0.04 


A AC 

0.05 


A AQ 

0.09 


C\ AC 

0.05 


A AC 

0.05 


4-5 


0.04 


0.03 


0.06 


0.06 


A AC 

0.05 


0.02 


0.04 


5-6 


0.03 


0.03 


0.05 


0.06 


0.02 


0.01 


0.04 


6-7 


0.01 


0.01 


0.04 


0.03 


0.01 


0.02 


0.03 


7-8 


0.02 


0.02 


0.05 


0.02 


0.01 


0.02 


0.03 


8-9 


0.02 


0.01 


0.03 


0.02 


0.01 


0.02 


0.02 


9-10 


0.01 


0.01 


0.01 


0.02 


0.01 


0.01 


0.01 


10-11 


0.02 


0.02 


0.00 


0.01 


0.01 


0.01 


0.01 


11-12 


0.01 


0.01 


0.01 


0.01 


0.01 


0.02 


0.01 


Continxiing 


0.07 


0.07 


0.12 


0.10 


0.09 


0.06 


0.09 



D-6 



ERIC 



Exhibit D.5 

Estimates of Persistence for All New Education Clients 



1 
1 


Z 


o 
J 


A 

't 


n 


D 


7 
/ 


Months of 
Activity 
Between 
Enrollment 

and 
Termination 


Number 
Active at 
the Start 
of Month 


Number 
Leaving 
Ehmng the 
Month 


L rupurQOii 
of Those 
Starting 

the Month 
who Left 

Month 


Total Client 
Months 
Logged 

During this 
'N/fonth 


Aggregate 
Client 
Months 

Expected at 
Start of 
Month 


Mean 
Number of 

Months 
Remaining 
at Start of 

Month 


0-1 


100,000 


32,552 


0.33 


83,724 


544,784 


5.45 


1-2 


67,448 


1130 


0.17 


61,833 


461,060 


6.84 


2-3 


56,218 


10,009 


0.18 


51,213 


399,228 


7.10 


3-4 


46,209 


5,048 


0.11 


43,685 


348,014 


7.53 


4-5 


41,161 


4,410 


0.11 


38,956 


304,330 


7.39 


5-6 


36,751 


4,339 


0.12 


34,582 


265,374 


7.22 


6-7 


32,412 


2,587 


0.08 


31,119 


230,793 


7.12 


7-8 


29,825 


3,875 


0.13 


27,888 


199,674 


6.69 


8-9 


25,951 


2,765 


0.11 


24,568 


171,786 


6.62 


9-10 


23,185 


1,887 


0.08 


22,242 


147,218 


6.35 


10-11 


21,299 


1,263 


0.06 


20,667 


124,976 


5.87 


11-12 


20,036 


1,899 


0.09 


19,086 


104,309 


5.21 


12-13 


18,136 


1,932 


0.11 


17,171 


85,223 


4.70 


13-14 


16,205 


5,115 


0.32 


13,647 


68,052 


4.20 


14-15 


11,089 


1,193 


0.11 


10,493 


54,405 


4.91 


15-16 


9,896 


976 


0.10 


9,408 


43,912 


4.44 


16-17 


8,920 


714 


0.08 


8,563 


34,504 


3.87 


17-18 


8,206 


1,426 


0.17 


7,493 


25,942 


3.16 


Continuing 


6,780 
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Exhibit D.6 

Estimates of Persistence for All New ESL Education Clients 



i 


/ 


Q 

J 




c 
D 


D 


7 
/ 


Months of 
Activity 
Between 

Enrollment 
and 

Termination 


Number 

Active at 
the Start 
of Month 


Number 
Leaving 
During the 
Month 


1 ruL/uriKJii 

of Those 
Starting 
the Month 
who Left 

L^UL xll^ ULLC 

Month 


Total Client 
Months 
Logged 

During this 


Aggregate 
Client 
Months 

Expected at 
Start of 
Month 


Mean 
Number of 

Months 
Remaining 
at Start of 

Month 


0-1 


100,000 


24;233 


0.24 


87,883 


678,836 


6.79 


1-2 


75,767 


10,182 


0.13 


70,676 


590,952 


7.80 


2-3 


65,585 


8,765 


0.13 


61,202 


520,277 


7.93 


3-4 


56,820 


4,863 


0.09 


54,388 


459,074 


8.08 


4-5 


51,956 


3,978 


0.08 


49,967 


404,687 


7.79 


5-6 


47,978 


4,719 


0.10 


45,619 


354,719 


7.39 


6-7 


43,260 


2,363 


0.05 


42,078 


309,100 


7.15 


7-8 


40,897 


5,021 


0.12 


38,386 


267,022 


6.53 


8-9 


35,875 


. 3,229 


0.09 


34,261 


228,636 


6.37 


9-10 


32,646 


2,541 


0.08 


31,376 


194,376 


5.95 


10-11 


30,105 


1,612 


0.05 


29,299 


163,000 


5.41 


11-12 


28,493 


2,566 


0.09 


27,210 


133,701 


4.69 


12-13 


25,927 


2,775 


0.11 


24,539 


106,491 


4.11 


13-14 


23,151 


9,311 


0.40 


18,496 


81,952 


3.54 


14-15 


13,840 


1,412 


0.10 


13,134 


63,457 


4.59 


15-16 


12,428 


1,276 


0.10 


11,790 


50,323 


4.05 


16-17 


11,152 


865 


0.08 


10,719 


38,534 


3.46 


17-18 


10,287 


1,790 


0.17 


9,392 


27,814 


2.70 


Continuing 


8,496 
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Exhibit D.7 

Estimates of Persistence for All New ABE Education Clients 



1 


2 


3 


4 


tr 
D 


6 


/ 


Months of 

UL V X I Y 

Between 
Enrollment 

and 
Termination 


Active at 
the Start 
of Month 


NIuTTiHpr 
Leaving 
During the 
Month 


Proportion 
of Those 
Starting 
the Month 
who Left 
Liunng tne 
Month 


Total Client 
Months 
Logged 

During this 
ivionui 


Aggregate 
Client 
Months 

Expected at 
Start of 
Month 


Mean 
Number of 

Months 
Remaining 
at Start of 

Month 


0-1 


100,000 


37,446 


0.37 


81,277 


446,161 


4.46 


1-2 


62,554 


12,299 


0.20 


56,404 


364,884 


5.83 


2-3 


50,255 


11,178 


0.22 


44,666 


308,480 


6.14 


3-4 


39,076 


5,840 


0.15 


36,157 


263,814 


6.75 


4-5 


33,237 


5,578 


0.17 


30,448 


227,657 


6.85 


5-6 


27,659 


4,302 


0.16 


25308 


197,210 


7.13 


6-7 


23357 


3,089 


0.13 


21,812 


171,702 


7.35 


7-8 


20,268 


2,653 


0.13 


18,941 


149,890 


7.40 


8-9 


17,615 


2,602 


0.15 


16,314 


130,949 


7.43 


9-10 


15,012 


1,462 


0.10 


14,281 


114,635 


7.64 


10-11 


13,550 


1,015 


0.07 


13,042 


100,354 


7.41 


11-12 


12,535 


1,153 


0.09 


11,958 


87,311 


6.97 


12-13 


11,382 


1,043 


0.09 


10,860 


75,353 


6.62 


13-14 


10339 


855 


0.08 


9,911 


64,493 


6.24 


14-15 


9,483 


955 


0.10 


9,006 


54382 


5.76 


15-16 


8,528 


583 


0.07 


8,237 


45376 


5.34 


16-17 


7,945 


500 


0.06 


7,695 


37,339 


4.70 


17-18 


7,446 


1,327 


0.18 


6,782 


29,643 


3.98 


Continuing 


6,119 
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Exhibit D.8 

Estimates of Persistence for All New ASE Education Clients 



1 

1 




o 
O 


4 


c 

D 


O 


/ 


Months of 
Activity 
Between 
Enrollment 

and 
Termination 


Number 
Active at 
the Start 
of Month 


Number 
Leaving 
During the 
Month 


L ropomon 
of Those 
Starting 

the Month 
who Left 

iyuruig uie 
Month 


Total Client 
Months 
Logged 

During this 
jYionin 


Aggregate 
Client 
Months 

Expected at 
Start of 
Month 


Mean 
Number of 

Months 
Remaining 
at Start of 

Month 


0-1 


100,000 


43,805 


0.44 


78,097 


378,979 


3.79 


1-2 


56,195 


12,286 


0.22 


50,052 


300,881 


5.35 


2-3 


43,909 


11339 


0.26 


38,239 


250330 


5.71 


3-4 


32^69 


4,877 


0.15 


30,131 


212391 


6.53 


4-5 


27,692 


4,355 


0.16 


25314 


182,460 


6.59 


5-6 


23337 


3,614 


0.15 


21330 


156,946 


6.73 


6-7 


19,723 


2,613 


0.13 


18,417 


135,416 


6.87 


7-8 


17,110 


2,691 


0.16 


15,765 


116,999 


6.84 


8-9 


14,419 


2,077 


0.14 


13380 


101,235 


7.02 


9-10 


12,342 


1,051 


0.09 


11,816 


87,854 


7.12 


10-11 


1131 


833 


0.07 


10,874 


76,038 


6.73 


11-12 


10,458 


1,253 


0.12 


9,831 


65,164 


6.23 


12-13 


9,205 


1,054 


0.11 


8,677 


55332 


6.01 


13-14 


8,150 


700 


0.09 


7,800 


46,655 


5.72 


14-15 


7,450 


944 


0.13 


6,978 


38,855 


5.22 


15-16 


6,506 


726 


0.11 


6,143 


31,877 


4.90 


16-17 


5,780 


602 


0.10 


5,479 


25,735 


4.45 


17-18 


5,178 


871 


0.17 


4,742 


20,256 


3.91 


Continuing 


4,307 
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Exhibit D,9 

Mean Number of Weeks of Enrollment, by Month of Enrollment, Region, and Component 

{N = 15,870) 



Region and 
Component 



Mean Weeks of Enrollment for Each Month of Enrollment 




Note: Number of weeks of enrollment is defined as the number of weeks that elapsed between initial instruction and final departure of the 
client from the program. 
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APPENDIX E 
Methodology of the Learning Gains Study 

(A Supplement to Chapter 3) 




Methodology of the Learning Gains Study 



A primary aim of the learning gains analysis was to estimate the effects of instruction 
by determining whether student achievement was related to attendance in adult education 
(e.g., total hours of instruction across all instructional settings). In order to maximize 
program cooperation with respect to the evaluation's primary objective of describing and 
estimating the national Adult Education Program population, a design trade-off was made 
in concentrating data collection on program participation while making the reporting of 
achievement test data volvmtary. Since greater research control over the collection of 
learning outcome data was believed to be xinfeasible (e.g., through the use of a quasi- 
experimental design), methods of statistical control were needed to attain the objectives of 
the analysis. A regression framework was therefore used to assess the effects of attendance 
(and other factors) on student achievement in adult education. 

Method 

Under the regression framework used, posttest performance was the outcome of 
interest, and the regression coefficient for total hours of instruction represented the overall 
effect of instruction. The validity of causal inferences based on this approach, however, 
depended on controlling for other variables related to the posttest. This in turn, depended 
on specifying an appropriate regression model that included both the relevant variables and 
a causal analysis strategy that properly controlled for intervening and extraneous variables. 

Design 

Indicators of student attendance were viewed as pivotal to the learning gains c^ialysis 
because the logic of the evaluation was that literacy outcomes should be a positive function 
of student "persistence" in the program (i.e., hours of instruction). In operationalizing the 
logic of the evaluation, student persistence was measured by hours of instruction between 
a pretest and posttest, and the effect of persistence on student achievement (i.e., posttest 
performance) was estimated using the ordinary least squares (OLS) linear multiple 
regression model. The analysis strategy statistically controlled for the influence of other 
independent variables not consequent to posttest performance (e.g., student backgrovind, 
motivation, ability, and variations in program resources). 

It was important in specifying an appropriate regression model to identify predictors 
relevant to posttest performance and to consider the order in which predictors were entered 
in the regressions. This appreciation argued for the use of a hierarchical analysis model in 
which a sequence of seven blocks of predictors were entered cumulatively into the 
regression equation for each of three models (one each for ESL, ABE, and ASE). The choice 
of the hierarchical strategy^ was based on the logic of causal priority (see, for example. 



^ Predictors entered in later blocks should not be plausible causes of prior variables entered in earlier 
blocks. For example, motivation for enrollment in adult education is not a plausible cause of a student's 
age, but age can plausibly influence one's motivation for enrolling in adult education. 
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Cohen and Cohen [1983]), and the seven block sequence was derived from exploratory 
analyses. 

The Data Base 

The program-level data used in the national evaluation of adult education programs 
were collected through two mail questionnaires (the Universe Survey and the 
Comprehensive Program Profile) during the 1990-91 school year from a nationally 
representative sample of 131 adult education programs. This initial phase of the study was 
used to select a sample of programs for longitudinal analysis. The longitudinal phase of the 
evaluation began in the spring of 1991 and ended in the fall of 1993, during which time 
client-level data were collected from a sample of 110 programs identified from the initial 
sampling frame. The client intake record and the client update record were used to collect 
client-level data over 19 monthly reporting periods for new students who eruroUed in adult 
education between April 1991 and April 1992; both types of data records were completed 
by local adult education personnel based on project records. The client intake record 
provided demographic and backgroimd information on new clients and the cUent update 
record provided data on student attendance, instruction, and standardized achievement test 
scores. The final data file for the learning gains analysis contained information on 18,351 
students from 110 local projects for whom their intake and update data were reasonable 
complete. 

Programs had the option of reporting no test data at all, and about half of the total 
client sample chose this option. For the other half of the client sample, pretest data were 
reported primarily for the Test of Adult Basic Education (TABE) and the Comprehensive 
Adult Student Assessment System (CASAS). However, we ultimately obtained complete 
attendance data and matched pretest-posttest TABE or CASAS scores for only about 1,900 
students from 65 local projects. The ESL students were tested almost exclusively with the 
CASAS, whereas the ABE and ASE students were tested primarily with the TABE. Once the 
final analysis groups were defined on the basis of four criteria used in screening the test 
data, the effective analysis sample was further reduced to approximately 1,300 students. 
Issues related to the quality of the test score data and the determination of the final analysis 
groups are discussed later in this appendix. 

The Outcome Measures 

Student learning outcomes were measured using posttest scale scores from the 
reading subtests of the TABE and CASAS. Pretest scores were used as control variables. 
CASAS scores were used to measure learning outcomes for the ESL sample, whereas TABE 
scores were used for the ABE and ASE analyses. Although we provided guidelines to local 
projects regarding when to pretest and posttest, state and local practices appear to have 
governed test administration in most projects. The questionable validity of much of the 
reported test data further exacerbated the problem of sample size adequacy. 

Developed and field-tested in California, the CASAS reading test was designed with 
a competency-based, life-skills orientation to literacy instruction in adult education (Rickard, 
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1988). CASAS scale scores range from 150 to 275 and have a criterion-referenced 
interpretation (see Sticht [1990] for technical details). 

The TABE reading subtest used in the learning gains analysis was designed by 
CTB/McGraw-HiU to measure comprehension of reading passages; subtest items assess 
recall, recognition, inference, and evaluation skills. The content of the TABE was based on 
a national review of curriculum guides, published texts, and instructional programs used 
in adult education.^ TABE scale scores on the reading comprehension subtest range from 
500 to 823; the grade equivalent is used in the learning gains analysis to provide norm- 
referenced interpretations of scale scores. TABE grade-equivalents were developed by 
CTB/McGraw-HiU through an equating of the TABE (Forms 5 and 6) with the California 
Achievement Test (Form E).^ 

The Predictor Variables 

The independent variables used in the final analysis are described and defined below 
within the context of the seven-block hierarchical analysis strategy.^ The factors assessed 
in the regression analysis are highlighted; variable names entered in the regression analyses 
are listed in capital letters in parenthesis. 

Block 1: Student Background. Race was measured as five dummy variables (1 if yes 
for American INDIAN, ASIAN, BLACK, HISPANIC, or WHITE, otherwise 0), with the 
dominant group for a given placement level serving as the reference category. For ESL 
placement, Hispanic was the reference group; for ABE and ASE placement, white was the 
reference group. Sex was measured as a single dummy variable (MALE) with female as the 
reference group. Age was measured as a continuous variable (AGE) derived from client 
birth date information. Educational attainment was measured as years of school completed 
(SCHOOLN) prior to enrollment in adult education, coded on an 8-point ordinal scale. 

Block 2: Enrollment Motivation. Whether or not a student was required to attend 
adult education was measured as a single dummy variable (coded 1 if the client was 
REQUIRED by an em;)loyer or another program to attend adult education, otherwise 0). 



^ See TABE (Fonns 5 and 6) Test Coordinator's Handbook; CTB/McGraw-Hill, 1987. 

^ See TABE (Forms 5 and 6) Technical Report; CTB/McGraw-HUl 1987. See also Table 71 of the 
TABE Norms Book which provides scale score to grade equivalent conversions. 

^ For a number of reasons, some independent variables examined in exploratory analyses were 
dropped from the final analysis. These reasons included nonsignificant relationships between the 
independent and dependent variables, redundancy among independent variables (i.e., high co-linearity), 
truncated variance, and lack of interpretabiUty. For hirther detail, see the glossary annexed to this 
appendix. 

^ The eight point ordinal scale is as follows: 1 = none; 2 = M years; 3 = 5-8 years; 4 = 9 years; 
5 = 10 years; 6 = 11 years; 7 = 12 years; 8 = more than 12 years. 
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Block 3: Entering Ability. Reading pretest scale scores were used to measxire 
entering achievement levels and were intended to represent client reading ability at the 
beginning of a client's program of instruction. CASAS reading pretest scores (CASPRE) 
represented entering ability for ESL clients, and TABE reading pretest scores (TABEPRE) 
were used for ABE and ASE students. 

Block 4: Program and Staffing Resources. Program curricultun (DESIGN) was 
measured on a 5-point scale from very individualized to very structured. Support services 
were measured as a dummy variable (SUPPORT2), coded 1 if used and 0 otherwise. Full- 
time staff was measured as a dummy variable (FTSTAFF), coded 1 if the program had at 
least one full-time administrator and one fuE-time teacher, 0 otherwise. Professional 
commitment, an indicator of teacher experience, was measured as a dummy variable 
(COMMIT) and coded 1 if either of the following two conditions were present: a majority 
of the program's instructional staff had more than 3 years of teaching experience in adult 
education, or at least one teacher was certified in adult education. Othei"wise, the 
professional commitment variable was coded 0. Cost per student hour of instruction was 
measxired on a 3-point scale (COST) with a score of 2 representing average cost; scores of 
1 or 3 reflect costs per hour of instruction at least one standard deviation below or above 
the mean of the national sample, respectively. 

Block 5: Class Size. Measured as the mean class size (MEANCLAS) across 
instructional environments for a given student during a given reporting period. 

Block 6: Class Attendance. Student participation in four principal instructional 
environments was measured by the number of hours of instruction received between the 
pretest and posttest in each of the following: classroom instruction only (CTEACHR for 
ESL students and TTEACHR for ABE and ASE students), lab only (CLABHR for ESL 
students and TLABHR for ABE and ASE students), independent study only (CINDHR for 
ESL students and TINDHR for ABE and ASE students), and class plus lab (CTCHLBHR for 
ESL students and TTCHLBHR for ABE and ASE students). In addition, the intensity of 
instructional attendance was measured by the mean number of instructional hours a student 
received per week between the pretest and the posttest (CASHRWK for ESL students and 
TABEHRWK for ABE and ASE students). 

Block 7: Total Hours of Instruction. The sum of a student's instructional hours 
between pretest and posttest across instructional environments measured total instructional 
time (CASHRS for ESL students and TABEHRS for ABE and ASE students). 

Determining the Analysis Groups 

Exploratory analyses resulted in the specification of inclusion rules for membership 
in the final analysis groups. These screening criteria are summarized below. 
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• Known placement in ESL, ABE, or ASE as indicated on the first client update 
record.^ 

• Matched CASAS or TABE pretest and posttest reading achievement scale 
score. 

• At least 1 hour of instruction between the pretest and posttest. 

• Evidence of test content validity relative to the CASAS or TABE as indicated 
by a program's score on item 15a of the Comprehensive Program Profile, 
which measured curriculum emphasis on a 5-point scale. 

The application of these four screening criteria generated six analysis groups. 
However, three of these groups were eliminated from further consideration because of 
sample size limitations.^ The three remaining final analysis groups are listed below with 
sample sizes as noted. 

1. ESL-CASAS N = 657 

2. ABE-TABE N = 192 

3. ASE-TABE N = 454 



Total sample N = 1303 

Extensive exploratory analyses had indicated the presence of floor and ceiling effects, 
particularly the latter. Ceiling effects were particularly pervasive on the TABE and had 
proved to be the major cause of heteroskedasticity in preliminary regression analyses 
involving the ABE and ASE groups, thus violating a major validity assumption of the OLS 
linear regression model (i.e., constant error variance). In addition, it was also evident that 
the validity of the pretest as a baseline measure was questionable for many clients who had 
generated a considerable number of instructional hours prior to the first administration of 
the CASAS or the TABE. This was especially evident in the case of ESL students. Thus, we 
felt it was necessary to impose exclusion rules related to floor and ceiling effects, and 
limitations on the number of instructional hours that would be permitted prior to the 
pretest. 

Cleaning the TABE data for floor and ceiling effects was hampered somewhat by the 
inaccuracy of the information that programs reported for the test forms they administered 
to students. Consequently, efforts were concentrated on limiting TABE ceiling effects since 
this was the major threat to the validity of the test scores. Information on the admiiustration 



^ In the total study sample, program placement was unknown in 431 cases. 

^ The three analysis samples eliminated from hirther consideration included an ESL-TABE group (n 
= 10), and ABE-CASAS group (n = 121) and an ASE-CASAS group (n = 190). 
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of the CASAS was more complete and allowed for a more comprehensive edit of both floor 
and ceiling effects in the CASAS data. 



In resolving the problem of invalid baseline measures, criteria were established 
limiting the number of instructional hours that would be allowed prior to the administration 
of a pretest. In the case of ABE and ASE students, a pretest was considered invalid if there 
were more than 6 hours of instruction prior to adrtiLnistration of the TABE pretest. This 
decision meant that the pretest had to be administered generally during the first week of 
ABE and ASE classes. After consulting with the developers of the CASAS, we decided to 
accept their recommendation for allowing up to 50 hours of ESL instruction prior to the 
administration of the CASAS pretest. This decision meant that the CASAS pretest generally 
had to be administered within the first month of ESL instruction. Empirical tests of the rules 
limiting the number of hoxirs of instruction prior to a pretest indicated that the constant 
error variance assumption of the linear regression model would again have been jeopardized 
if the pretest exclusion rules had not been imposed. 

Cleaning the test data considerably reduced the number of cases for final analysis. 
The single greatest threat to the validity of TABE scores involved posttest ceiling effects. For 
the CASAS data, the single greatest problem involved invalid baseline measures as reflected 
by those ESL students who received in excess of 50 hours of instruction prior to being 
administered the CASAS pretest. The weighted final analysis samples were distributed as 
follows after editing for invalid test scores. 



1. 
2. 

3. 



ESL-CASAS N = 349 
ABE-TABE N = 111 
ASE-TABE N = 154^ 



Final analysis 
total sample: 



N = 614 



^ In conducting the analysis, one case from the ASE-TABE group proved to be an influential outlier 
and was consequently dropped from the final model for this group. 
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The Analysis of Learning Gains 



OLS regressions were conducted with each of the three final samples using the 7- 
block hierarchical analysis described earlier. Development of the three regression models 
was guided by the following principles: (a) maximization of the R-Square statistic,^ and (b) 
plausibility of independent variable relationships with the outcome variable. One-sample 
paired t-tests were also computed to examine the significance of gain scores for each of the 
three analysis groups. In addition, analysis of covariance was employed to examine the 
significance of differences in adjusted posttest means associated with the three principal 
instructional environments measured as categorical variables (classroom-only, lab-only, and 
classroom plus lab); significant predictors of posttest performance in each of the three 
regression models were used as covariates in the respective analysis of covariance 
applications for ESL, ABE, and ASE. The sampling weights employed in these analysis were 
corrected for sample size and designed to compensate for the nonrandom composition of 
the final analysis groups.^^ 

The analysis samples are described below in exhibits E.2-E.4 using the predictors 
entered in each analysis block of the respective regression models. These tables compare 
the means and standard deviations of predictors in the analysis samples (using the revised 
sampling weights) with the total ESL, ABE, and ASE study samples (using the original 
sampling weight). In general, estimates from the analysis samples and the total study 
samples are very close. 

Comparable information is lacking on attendance indicators for the total samples in 
exhibits E.2-E.4 because these variables (i.e., total hours and hours of instruction in specific 
instructional environments) were constructed within the context of instructional time 
between pretest and posttest. However, information is available for both the analysis 
samples and the total study sample with respect to the total hours of program attendance 
throughout the study. As noted in Exhibit E.l below, program attendance was generally 
greater for members of the analysis samples compared to their counterparts in the total 
study sample. 



^ K-Square is a Goodness of Fit statistic for estimating how well the model fits the data; it indicates 
the proportion of variance in the dependent variable (e.g., posttest perfomiance) accounted for by the 
predictors entered into an OLS regression equation. 

Using estimated probabilities for selection into the final analysis sample and corrections for sample 
size, weights were developed for each of the three final analysis groups which adjusted the sampling 
weight used for the total study sample (DIWAIT) in order to generate more accurate estimates from the 
analysis samples for both descriptive and inferential purposes. 
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Exhibit E.1 

Mean Hours of Program Attendance Throughout the Study 
for ESL, ABE, and ASE Students 



Program 


Learning Gains 


Total 


Component 


Analysis Sample 


Study Sample 


ESL 


288.5 hours 


223.8 hours 


ABE 


141.8 hours 


83.9 hours 


ASE 


95.2 hours 


68.0 hours 



The variable names used in exhibits E.2 to E.4 are more fully explained in the 
glossary at the end of Appendix E. Variables that were examined during exploratory 
analyses but which were dropped from the final analysis also are listed in the glossary. 

The ESL Sample. Hispanic, the reference group for race in the ESL sample, was not 
entered into the ESL regression equation; it is displayed in exhibit E.2 for descriptive 
purposes only. For both the ESL final analysis sample and total ESL sample, the mean value 
of SCHOOIJNf converts to 10 years of school completed prior to enrolling in adult education. 
Similarly, the mean scale score for DESIGN in both groups indicates that ESL instruction is 
generally described by program staff as individualized. In addition, cost per student hour 
(COST) is low in both the final analysis sample and in the total ESL sample. 

Hours of instruction for ESL classrooms (CTEACHR) was highly colinear with total 
hours of instruction (CASHRS) and was therefore not entered into the ESL regression 
equation. Independent study (CINDHR) also was not entered, in this case because ESL 
students typically do not enroll only for independent study and because it was highly 
colinear with INDIAN, who do primarily enroll in ESL through independent study. The 
CTEACHR AND CINDHR variables are therefore not displayed in exhibit E.2. 
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Exhibit E.2 
The Weighted ESL Final Analysis Sample 
Compared with the Weighted Total ESL Study Sample 



Analysis 




ESL Analysis Sample 


Total ESL Sample 


Block 


Predictors 


(N = 


349) 


(N = 6,185) 






Mean 


SD 


Mean 


SD 


Student 


INDIAN 


.009 1 


.093 


.007 


.083 


background 


ASIAN 


.210 


.408 


.201 


.401 




BLACK 


.010 


.098 


.013 


.111 




WHITE 


.068 


.253 


.060 


.237 




HISPANIC 


.703 


.458 








MALE 




499 


.462 


.499 




AGE 


30.464 


10.852 


30.189 


11.179 






5.424 


2.155 


5.178 


2.161 


Motivation 


REQUIRED 


.092 


.281 


.094 


.292 


Ability 


CASPRE 


206.917 


15.404 


206.851 


15.490 


Program 


DESIGN 


1.848 


1.157 


2.016 


1.105 


and staff 


SUPPORT2 


.138 


.346 


.117 


.322 


resources 














r IblArr 


.879 


.326 


.866 


.340 




COMMIT 


.963 


.189 


.914 


.281 




COST 


1.314 


.473 


1.513 


.679 


Class size 


MEANCLAS 


35.966 


12.902 


33.789 


12.294 


Class 


CASHRWK 


8.835 


2.705 






attendance 


CLABHR 


5.486 


25.872 








CTCHLBHR 


23.118 


48.355 






Total hours 


CASHRS 


120.055 


113.792 
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The ABE Sample. White, the reference group for race in the ABE sample, was not 
entered into the ABE regression equation; it is displayed in exhibit E.3 for descriptive 
purposes only. For both the ABE final analysis sample and total ABE sample, the mean 
value of SCHOOLN converts to approximately 10 years of school completed prior to 
enrollment in adult education. Similarly, the mean scale score for DESIGN in both groups 
indicates that ABE instruction is generally described by program staff as individualized. In 
addition, cost per student hour (COST) is in the average range for both the final analysis 
sample and the total ABE sample. 

In the ABE analysis sample, hours of instruction in the teacher plus lab environment 
(TTCHLBHR) was highly colinear with total hours of instruction and was not entered into 
the ABE regression equation; it is therefore not displayed in exhibit E.3, The ratio of full- 
time teachers to total instructional staff (STAFFRAT) was foimd to be highly co-linear with 
both full-time staff (FTSTAFF) and class size (MEANCLAS); it was therefore not entered into 
the ABE regression model Exploratory analyses indicated that it was the highly 
individualized aspect of the curriculimi design variable that was predictive of ABE student 
achievement; therefore, a duixuny variable representing highly individualized curriculxmi 
design (TAILOR2) was entered in the ABE regression model The DESIGN variable is 
displayed in exhibit E.3 rather than TAILOR2 for descriptive and comparative purposes. 

The ASE Sample. White, also the reference group for race in the ASE sample, was 
not entered into the ASE regression equation; it is displayed in exhibit E.4 for descriptive 
purposes only. For both the ASE final analysis sample and total ASE sample, the mean 
value of SCHCX)LN converts to approximately 10 years of school completed prior to 
enrollment in adult education. Similarly, the mean scale score for DESIGN in both groups 
indicates that ASE instruction is generally described by program staff as individualized. In 
addition, cost per student hour (COST) is in the average range for both the final analysis 
sample and the total ASE sample. 

In the ASE analysis sample, the variable measuring classroom instruction hours 
(TTEACHR) was highly colinear with total instructional hours (TABEHRS). The class hours 
variable was therefore not entered into the ASE regression model and it is not displayed in 
exhibit E.4. In addition, the full-time staff variable (FTSTAFF) was foimd to be highly co- 
linear with both the program design variable (DESIGN) and the proportion of full-time 
teachers on the instructional staff (STAFFRAT). Because exploratory analyses had shown 
that it was the individualized aspect of the design variable that was most related to student 
achievement, a dummy variable version of curriculum design representing individualized 
instruction (TAILOR) was entered and STAFFRAT was dropped in favor of FTSTAFF. The 
DESIGN variable is displayed in exhibit E.4 rather than TAILOR for descriptive and 
comparative purposes. 
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Exhibit E3 
The Weighted ABE Final Analysis Sample 
Compared with the Weighted Total ABE Study Sample 



Analysis 




ABE Analysis Sample 


ABE Total Sample 


Block 


Predictors 


(N = 


111) 


(N = 4,468) 






Mean 


SD 


Mean 


SD 


Student 


INDIAN 


.048 


.215 


.028 


.164 


background 


ASIAN 


.000 


.000 


.019 


.135 




BLACK 


.293 


.457 


.255 


.436 




HISPANIC 


.099 


.300 


.098 


.298 




WHITE 


.560 


.499 








MALb 


4.10 

.*±i.U 


494 


.399 


.490 




AGE 


32.182 


•too 

13.300 


29.346 


11.531 




bCrlUULIM 


4.625 


1.343 


4.757 


1.493 


Motivation 


REQUIRED 


.142 


.347 


.146 


.353 


Ability 


TABEPRE 


727.784 


38.206 


725.880 


72.273 


Program and 


DESIGN 


1.892 


1.161 


2.067 


1.174 


staff 


SUPPORT2 


.662 


.475 


.348 


.477 


resources 












r ibl Arr 


.656 


A77 


.188 


.242 




COMMIT 


.753 


.434 


.733 


.443 




COST 


2.228 


.455 


2.356 


.593 


enlace 


MEANCLAS 


16.590 


16.747 


22.605 


29.571 


Class 


TABEHRWK 


6.022 


4.037 






attendance 


TTEACHR 


30.809 


41.252 








TLABHR 


15.398 


40.978 








TINDHR 


5.237 


19.871 






Total hours 


TABEHRS 


83.556 


72.539 
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Exhibit E.4 
The Weighted ASE Final Analysis Sample 
Compared with the Weighted Total ASE Study Sample 



Analysis 




ASE Analysis Sample 


ASE Total Sample 


Block 


Predictors 


(N = 


154) 


(N = 7,267) 






Mean 


SD 


Mean 


SD 


Student 


INDIAN 


.055 


.229 




1Q9 


backerotind 


ASIAN 


.021 


.143 


.018 


.133 




BLACK 


.149 


.357 


.156 


.363 




HISPANIC 


.046 


.211 


.097 


.296 




WHITE 


.730 


.445 


.D/y 


Ab/ 




\A ATT? 






.380 


.485 




AGE 


25.685 


8.211 


27.409 


11.045 




COX-IOOT NT 


5.201 


1.192 


5.069 


1.302 


Motivation 




.274 


.440 


.114 


.317 


Ability 


TABEPRE 


754.862 


18.849 


763 222 


33 263 


Program and 


DESIGN 


2.454 


1.254 


2.050 


1.109 


staff 


SUPPORT2 


.407 


.493 


.303 


.460 


resources 














FTSTAFF 


.396 


.484 


.453 


.498 




COMMIT 


.527 


.495 


.690 


.462 




COST 


2 107 


540 


2.349 


.639 


Class size 


MEANCLAS 


22.179 


14.730 


23.404 


24.183 


Class 


TABEHRWK 


6.309 


3.783 






attendance 


TLABHR 


4.012 


14.436 








TTCHLBHR 


18.878 


40.357 








TINDHR 


.922 


7.019 






Total hours 


TABEHRS 


62.504 


61.677 
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statistical Tables in This Appendix 



Exhibits E.5 through E.ll summarize the statistical results discussed in chapter 3. 
Exhibit E.5 below displays the t-test results for the gain score findings presented in Exhibit 
3.3 of chapter 3. 

Exhibit E.5 
One Sample Paired t-test Results: 
Mean Differences between Pretest and Posttest 

for 

ESL, ABE, and ASE Students in the Final Analysis Groups 



Analysis 
Group 


N 


Mean 
Difference 


Standard Error 
of Mean 


t-Value (df) 


2-Taa 
Significance 


ESL-CASAS 


349 


5.06 


.80 


6.32 (348) 


.000 


ABE-TABE 


111 


15.70 


2.95 


5.33 (110) 


.000 


ASE-TABE 


154 


7.05 


1.39 


5.0*^- (153) 


.000 
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Exhibits E.6 through E.8 summarize the regression analysis findings regarding direct 
effects on client literacy outcomes for the ESL, ABE, and ASE components. In these exhibits, 
independent variables (i.e., predictors) are grouped by the analysis block in which they were 
entered in the regression. The unstandardardized regression coefficient (B) is displayed in 
the second column and the standard error associated with the unstandardized regression 
coefficient (SE B) is displayed in the third column for each predictor; the standardized 
regression coefficient (Beta) is displayed in the fourth column. The statistical significance 
of the F ratio for each predictor is displayed in the last column. All entries in Exhibits E.6 
through E.8 are rounded to three decimal places. 

Exhibit E.6 

Results of the ESL Regression Analysis Model: 
Direct Effects on Client Literacy Outcomes Measured by CASAS Reading Posttest Scores 



Predictor 


B 


SEB 


Beta 


Sienificance 


INDIAN 


1 o Co 




-.UUo 


.oZU 


A CT A XT 

AblAN 


-2.(JZZ 


i.ZUo 






BLACK 


3.190 


4.825 


.021 


.509 


WHITE 


1L363 


1.890 


.191 


.000 


MAT V 


.oyo 








AGE 


.050 


.047 


.036 


.289 


3CHOOLN 


.978 


.258 


.141 


.000 


REQUIRED 


-.221 


2.141 


-.004 


.917 


CASPRE 


.674 


.038 


.692 


.000 


DESIGN 


-.187 


.539 


-.014 


.728 


SUPPORT2 


2.603 


1.861 


.060 


.163 


FTSTAFF 


.174 


1.897 


.004 


.927 


COMMIT 


-1.721 


3.119 


-.022 


.581 


COST 


5.373 


1.545 


.170 


.001 


MEANCLAS 


.075 


.075 


.064 


.320 


CASHRWK 


-.762 


.210 


-.138 


.000 


CLABHR 


-.022 


.035 


-.037 


.533 


CTCHLBHR 


.009 


.012 


.030 


.426 


CASHRS 


.025 


.005 


.192 


.000 



N = 349 

Adjusted R Square = .70 
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Exhibit E.7 

Results of the ABE Regression Analysis Model: 
Direct Effects on Client Literacy Outcomes Measured by TABE Reading Posttest Scores 



Predictor 


B 


SE B 


Beta 


Sigruficance 


INDIAN 


4.769 


12.122 


.033 


.695 


BLACK 


-1.706 


5.113 


-.025 


.739 


HISPANIC 


-.199 


6.797 


-.002 


.977 


MALE 


-3.426 


4.184 


-.055 


.415 


AGE 


-.083 


.155 


-.035 


.595 


SCHOOLN 


-3.101 


1.811 


-.133 


.090 


REQUIRED 


-13.764 


5.653 


-.153 


.017 


TABEPRE 


.635 


.053 


.780 


.000 


T A TT f^T?0 




5 341 


.277 


.002 


bUl i UKlz 




4 621 


.020 


.783 


FTSTAFF 


16.681 


6.612 


.256 


.013 


COMMIT 


-9.419 


5.899 


-.131 


.114 


COST 


2.448 


5.233 


.036 


.641 


MEANCLAS 


-.020 


.140 


-.011 


.887 


TABEHRWK 


-.793 


.728 


-.104 


.279 


TTEACHR 


-.113 


.056 


-.150 


.046 


TLABHR 


-.131 


.058 


-.172 


.027 


TINDHR 


-.032 


.113 


-.020 


.778 


TABEHRS 


.060 


.037 


.140 


.112 



N= 111 

Adjusted R Square = .70 
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Exhibit E.8 

Results of the ASE Regresc .on Analysis Model: 
Direct Effects on Client Literacy Outcomes Measured by TABE Reading Posttest Scores 



Predictor 


B 


SE B 


Beta 


Significance 


INDIAN 


-6.859 


5.244 


-.110 


.193 


ASIAN 


18.393 


8.706 


.182 


.037 


BLACK 


-10.842 


3.686 


-.261 


.004 


HISPANIC 


-9.507 


5.658 


-.118 


.095 


MALE 


1.970 


2.408 


.059 


.415 


AGE 


.031 


.119 


.018 


.794 


SCHOOLN 


-1.452 


.olo 


-.iiy 


.077 


KbyUlKbD 


1.2/7 


2.0/0 


.(Joy 


coo 


TABEPRE 


.423 


.072 


.439 


.000 








077 
.u/ / 




SUPPORT2 


2.711 


2.630 


.092 


.305 


FTSTAFF 


-3.612 


4.153 


-.122 


.386 


COMMIT 


-9.359 


3.846 


-.324 


.016 


COST 


5.949 


2.794 


.217 


.035 


MEANCLAS 


.055 


.079 


.056 


.492 


TABEHRWK 


-.227 


.389 


-.060 


.560 


TLABHR 


.119 


.077 


.120 


.128 


TTCHLBHR 


.021 


.034 


.059 


.539 


TINDHR 


-.034 


.139 


-.017 


.806 


TABEHRS 


-.014 


.020 


-.058 


.503 



N= 154 

Adjusted R Square = .49 
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Exhibits E.9 through E.ll present the analysis of covariance results for adjusted 
posttest scores associated with ESL, ABE, and ASE enrollment in three types of instructional 
environments. 

Exhibit E.9 displays the results of the analysis of covariance for ESL clients 
participating in the three principal instructional environments, measured as categorical 
variables (lab-only, classroom plus lab, and classroom-only).^^ Covariates used to adjust 
ESL-<1ASAS posttest scores included CASAS pretest score, years of school completed, 
race /ethnicity (dummy variable for WHITE), cost per hour of instruction, and total hours 
of instruction. There was no statistically significant difference overall in adjusted CASAS 
posttest scores for ESL clients enrolled in the three types of environments (p = .61). Simple 
contrasts on adjusted posttest means between lab-only versus classroom-only (p = .33) and 
between classroom plus lab and classroom-only (p = .88) were also nonsignificant. 

Exhibit E.9 
Analysis of Covariance Results 
for ESL Participation in Three Ltistructional Environments: 
Contrasts of Lab-Only and Classroom-Plus-Lab with Classroom-Only 
on Adjusted CASAS Posttest Scores 



Instructional 


Observed 


Adjusted 


Sample 


Environment 


Posttest Mean 


Posttest Mean 


Size 


Lab-Only 


225 


216 


n= 14 


Class+Lab 


218 


218 


n = 41 


Class-Only 


211 


219 


n = 290 


Total 


212 


219 


N = 345 



The ESL independent study-only environment (n 
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= 4) was not included in this analysis. 
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Exhibit E.IO displays the results of the analysis of covariance for ABE clients 
participating in the three principal instructional environments, measured as categorical 
variables (lab-only, classroom plus lab, and classroom-only).^^ Covariates used to adjust 
ABE-TABE posttest scores included TABE pretest score, whether or not the client was 
reqtilred to enroll, curriculimi design (dummy variable for TAILOR2), and full-time staff. 
There was no statistically significant difference overall in adjusted TABE posttest scores for 
ABE clients enrolled in the three types of environments (p = .99). Simple contrasts on 
adjusted posttest means between lab-only versus classroom-only (p = .89) and between 
classroom plus lab and classroom-only (p = .99) were also nonsignificant. 

Exhibit E.10 
Analysis of Covariance Results 
for ABE Participation in Three Instructional Environments: 
Contrasts of Lab-Only and Classroom-Plus-Lab with Classroom-Only 
on Adjusted TABE Posttest Scores 



Instructional 


Observed 


Adjusted 


Sample 


Environment 


Posttest Mean 


Posttest Mean 


Size 


Lab-Only 


765 


748 


n = 14 


Class+Lab 


745 


749 


n = 36 


Class-Orily 


735 


749 


n = 51 


Total 


743 


749 


N = 101 



The ABE independent study-only environment (n = 10) was not included in this analysis. 
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Exhibit E.ll displays the results of the analysis of covariance for ASE clients 
participating in the three principal instructional environments, measured as categorical 
variables (lab-only, classroom plus lab, and classroom-only).^^ Covariates used to adjust 
ASE-TABE posttest scores included TABE pretest score, race /ethnicity (dummy variables for 
ASIAN and BLACK), cost per hour of instruction, and whether the program was 
characterized as "committed" or not. There was no statistically significant difference overall 
in adjusted TABE posttest scores for ASE clients eru-oUed in the three types of environments 
(p = .18). Simple contrasts on adjusted posttest means between lab-only versus classroom- 
only (p = .07) and between classroom plus lab and classroom-only (p = .57) were also 
nonsignificant. 

Exhibit E.11 
Analysis of Covariance Results 
for ASE Participation in Three Instructional Enviroiunents: 
Contrasts of Lab-Only and Classroom-Plus-Lab with Classroom-Only 
on Adjusted TABE Posttest Scores 



Instructional 


Observed 


Adjusted 


Sample 


Environment 


Posttest Mean 


Posttest Mean 


Size 


Lab-Only 


774 


768 


n = 16 


Class+Lab 


758 


763 


n = 40 


Class-Only 


760 


761 


n = 89 


Total 


761 


762 


N = 145 



" The ASE independent study-only environment (n = 9) was not included in this analysis. 
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Appendix E 

Glossary of Independent and Dependent Variables 



1. Variables Entered in the Final Analysis 



Variable Name 



Variable Label 



AGE 

MALE 

INDL\N 

ASIAN 

BLACK 

HISPANIC 

WHITE 

SCHOOLN 

REQUIRED 

CASPRE 
TABEPRE 

DESIGN 

TAILOR 

TAILOR2 

SUPPORT2 

FTSTAFF 

COST 

COMMIT 



Client age in years. (Interval) 

Dummy variable for client gender. 

Dummy variable for race: American Indian. 

Dummy variable for race: Asian. 

Dummy variable for race: black, not Hispanic. 

Dummy variable for race: Hispanic. 

Dummy variable for race: white, not Hispanic. 

Years of school completed. (Ordinal) 

Dummy variable for required to attend adult education. 

CASAS pretest scale score. (Interval) 
TABE pretest scale score. (Interval) 

Curriculum design: very individualized to structured. (Interval) 

Dummy variable for individualized instruction. 

Dummy variable for very iadividualized instruction. 

Dummy variable for use of support services. 

Dummy variable for presence of full-time staff. 

Cost per student hour of instruction. (Ordinal) 

Dummy variable for presence of 3 years' teaching experience or 

certification. 



MEANCLAS 



Average Class Size. (Interval) 



CASHRWK 

TABEHRWK 

TTEACHR 

TLABHR 

TINDHR 

TTCHLBHR 

CTEACHR 

CLABHR 

CINDHR 

CTCHLBHR 



Hours per week between CASAS pretest and posttest. (Interval) 
Hours per week between TABE pretest and posttest. (Interval) 
Class hours between TABE pretest and posttest. (Interval) 
Lab hours between TABE pretest and posttest. (Interval) 
Independent hours between TABE pretest and posttest. (Interval) 
Class plus lab hours between TABE pretest and posttest. (Interval) 
Class hours between CASAS pretest and posttest. (Interval) 
Lab hours between CASAS pretest and posttest. (Interval) 
Independent hours between CASAS pretest and posttest. (Interval) 
Class plus lab hours between CASAS pretest and posttest. (Interval) 
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CASHES Total hours between CASAS pretest and posttest. (Interval) 

TABEHRS Total hours between TABE pretest and posttest. (Interval) 

CASPOST CASAS posttest scale score. (Interval) 

TABEPOST TABE posttest scale score. (Interval) 



II. Variables Dropped from the Final Analysis 
Variable Name Variable Label 



DEGREE 

MARRIED 

SINGLE 

LOSTMATE 

WELFARE 

WOR 

USYEARS 

SPEKLAN 

READLAN 

WSPKENG 

EMPLOY 
BASIC 
LITERATE 
SELF 

SERVEINT 

ESLMOS 

ABEMOS 

ASEMOS 

STAFFRAT 

TEACHRAT 

DAY 

NTTE 

DAYNITE 

EMPHASIS 

CONT 

RATIOHRS 

TEACHER 

LAB 

NONTEACH 



Highest degree obtained (Ordinal) 

Dummy variable for marital status: currently married. 

Dummy variable for marital status: never married. 

Dummy variable: Widowed, separated, or divorced. 

Dummy variable for welfare status. 

Dummy variable for employment status. 

Years foreign-bom clients have lived in USA. (Interval) 

Dummy variables for language spoken in the home. 

Native language reading ability. (Ordinal) 

English language speaking ability. (Ordinal) 

Factor score for enrollment motivation: Employment. 

Factor score for enrollment motivation: basic skills. 

Factor score for enrollment motivation: literacy 

Factor score for enrollment motivation: self-esteem. 

Services integration (Ordinal) 
Number of months ESL classes held. (Interval) 
Number of months ABE classes held. (Interval) 
Number of months ABE classes held. (Interval) 
Ratio of full-time teachers to total teachers. 
Teacher-student ratio at the program level. (Interval) 
Dunrary variable: day class attendance only. 
Dununy variable: night class attendance only. 
Dununy variable: day and night class attendance. 
Indicator variable for content validity. 

Continuity of instruction between pretest and posttest. (Interval) 
Attendance rate measured in hours per week. (Interval) 
Dununy variable for classroom learning envirorunent. 
Dununy variable for CAI or learning lab envirorunent. 
Dununy variable for independent study or tutoring environment. 
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APPENDIX F 

Selected Updated Tables From Second Interim Report 
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The first three reports of the National Evaluation of Adult Education Programs 
were interim and particularly the second and third reports used data that were in 
constant flux. In this appendix we provide updated versions of selected tables from 
the Second Interim Report: Profiles of Client Characteristics. We have retained the 
original table numbering to facilitate cross-reference. Key tables from the third 
interim report were updated and presented in Chapter 2 of this report. Most of the 
tables in the first report are based on the program level surveys, and these data have 
not changed. 
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Exhibit 3.1 
Distribution of New ESL Clients by 
Racial/Ethnic Composition 
(N = 7,525) 



ixaClai/ CXXlTUv. Vjruup 




American Indian/ Alaskan Native 


<1 % 


Asian/Pacific Islander 


19 


Black, non-Hispanic 


4 


Hispanic 


69 


White, non-Hispanic 


7 


Total 


100 % 



Exhibit 3.2 

Native-Language Reading Ability and English-Speaking Ability, 
by Percent of New ESL Clients 
Who Speak a Language Other Than English at Home 

(N = 4,964) 



Self-Rated Ability 


Reading in Native 
Language 


Speaking English 


Not at ail 


1 % 


25 % 


Not well 


7 


62 


Well 


26 


11 


Very well 


66 


2 


Total 


100 % 


100 % 
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Exhibit 3.3 

The Relationship Between Self-Reported Native Language 
Reading Ability for New ESL Clients Who Speak a Language Other 

Than English at Home 
(N = 4,925) 



Very 
Well 



English-Speaking 
Ability 



Not 
Well 



Not At 

AM Not at Ail Not Well Well Very Well 



Note: Values in chart are mean scores on a seal with "Not at all" = 0 
and "Very Well" = 3 
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Exhibit 3.4 
Distribution of New ESL Clients 
by Age Group 
(N = 7,626) 



Age Group 


New ESL Clients 


16-21 


22% 


22-30 


39 


31-45 


29 


Over 45 


10 


Total 


100 % 



Exhibit 3.5 

Educational Attainment of New ESL Clients 
(N = 6,301) 



Highest Level of 
Education Attained 


New ESL Clients 


No high school diploma or GED 


50 % 


High school diploma or GED 


29 


Postsecondary degree 


21 


Total 


100 % 



Exhibit 3.6 
Distribution of New ESL Clients 
by Type of Agency Sponsoring the Program 
iN = 7,626) 



Type of Program Sponsor 


New ESL CHents 


Public school system 


84 % 


Commimity college 


12 


Private voluntary organization 


2 


Regional education service agency 


1 


Technical institute 


1 


Total 


100 % 
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Exhibits.? 

Distribution of New ESL Clients by Type of Community 

(N = 7,626) 



Type of Community 


Percent of ESL Clients 


Large city in major metro area 


54% 


Remainder of major metro area 


31 


Small metro area 


11 


Norunetro area 


4 


Total 


100% 



Note: Commiinity designations are based on 1990 census information plus responses to 
the Universe Survey on type of community served (item 3). Major metropolitan areas 
are defined as those having a population of 1.5 million or more; large cities in major 
metro areas as those having a population of 500,000 or more; and small metro areas as 
any commtinity located within a standard metropolitan area with a population of less 
than 1.5 million. 



Exhibit 3.8 
Distribution of New ESL Clients 
by Census Region 
(N = 7,626) 



Census Region 


New ESL Clients 


Northeast 


6% 


North Central 


9 


South 


13 


West 


72 


Total 


100 % 



Exhibit 3.9 

National Origin of New Clients by Instructional Program 

(N = 16,420) 



National Origin 


ESL 


ABE/ASE 


Overall 
Average 


Native-bom 


2% 


87% 


47% 


Foreign-bom 


98 


13 


53 


Total 


100 % 


100 % 


100 % 



Exhibit 3.10 

Distribution of New ABE/ASE Clients by Racial/Ethnic Composition 

(N = 14,176) 



Ethnic Group 


ABE 


ASE 


Overall 
Average 


American Indian/Alaskan Native 


2% 


4% 


3% 


Asian/Pacific Islander 


1 


2 


2 


Black, non-Hispanic 


39 


20 


28 


Hispanic 


12 


13 


13 


White, non-Hispanic 


46 


61 


54 


Total 


100 % 


100 % 


100 % 



Exhibit 3.11 
Racial/Ethnic Distribution of New Clients 
by Type of Instructional Program 
(N = 21,701) 



Ethnic Group 


ESL 


ABE/ASE 


Overall 
Average 


American Indian/Alaskan Native 


<1 % 


3 % 


2% 


Asian/Pacific Islander 


19 


2 


10 


Black, non-Hispanic 


5 


28 


17 


Hispanic 


69 


13 


38 


White, non-Hispanic 


7 


54 


33 


Total 


100 % 


100 % 


100 % 
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Exhibit 3.12 

Differences in Language Spoken in the Home 
for New ESL and ABE/ASE Clients 
(N = 14,090) 



Language 
Spoken in the Home 


ESL 


ABE/ASE 


Overall 
Average 


English 


4 % 


82 % 


42 % 


Other than English 


96 


18 


58 


Total 


100% 


100 % 


100% 



Exhibit 3.13 
Distribution of New ABE/ASE Clients 
by Age Group 
(N = 14,393) 



Age Group 


ABE 


ASE 


Overall 
Average 


16-21 


28% 


44 % 


37 % 


22-30 


30 


27 


28 


31-45 


31 


23 


27 


Over 45 


11 


6 


8 


Total 


100 % 


100 % 


100 % 



Exhibit 3.14 

Educational Attainment of New ABE/ASE Clients 
(N = 12,478) 



Highest Level of 
Education Attained 


ABE 


ASE 


Overall 
Average 


No high school diploma 


78% 


87% 


83 % 


High school diploma 


20 


10 


14 


Postsecondary degree 


2 


3 


3 


Total 


100 % 


100% 


100% 
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Exhibit 3.15 
Differences in Educational Achievement 
by New ESL and ABE/ASE Clients 
(N = 18,779) 



Highest Level of 
Education 


ESL 


ABE/ASE 


Overall 
Average 


No diploma 


50 % 


83 % 


68% 


High school diploma 


29 


14 


21 


Postsecondary degree 


21 


3 


11 


Total 


100% 


100 % 


100 % 



Exhibit 3.16 
Distribution of New ABE/ASE Clients 
by Type of Sponsoring Agency 
(N = 14,393) 



Type of Sponsor 


ABE 


ASE 


Overall 
Average 


Public school system 


59 % 


68 % 


64% 


Community college 


29 


20 


24 


Technical institute 


4 


6 


5 


Private voluntary 
organization 


6 


3 


4 


Regional education service 
agency 


2 


3 


3 


Total 


100 % 


100 % 


100 % 
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Exhibit 3.17 
Distribution of New ABE/ASE Clients 
by Type of Community 
(N = 14^93) 



Type of Commimity 


ABE 


ASE 


Overall 
Average 


Large city in major metro area 


13 % 


7% 


9 % 


Remainder of major metro area 


30 


26 


28 


Small metro area 


27 


33 


30 


Noimietro area 


30 


■ 34 


33 


Total 


100% 


100 % 


100 % 



Exhibit 3,18 
Distribution of New ABE/ASE Clients 
by Census Region 
(N = 14,393) 



Census Region 


ABE 


ASE 


Overall 
Average 


Northeast 


15% 


14% 


15% 


North Central 


29 


27 


28 


South 


49 


41 


44 


West 


7 


18 


13 


Total 


100 % 


100 % 


100 % 
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Exhibit 4.1 

Distribution of New Clients by Type of Sponsoring Agency 
and Program Component 
.(N = 20,718) 



Type of Sponsor 


ESL 


ABE 


ASE 


Average 


Local education agency 
(LEA) 




59 % 


68 % 


73 % 




12 


29 


20 


18 


Private voluntary 
organization 


2 


6 


3 


3 


Technical institute 


1 


4 


6 


3 


Regional education 
service agency 


1 


2 


3 


3 


Total 


100 % 


100 % 


100% 


100 % 



Exhibit 4.2 

Distribution of New Clients by Type of Sponsoring Agency 

and Race/Ethnicity 
(N = 20^73) 



Type of 
Sponsoring 
Agency 


American 
Indian/ 
Alaskan 
Native 


Asian 
Pacific 
Islander 


Black, 
non- 
Hispanic 


Hispanic 


White, 
non- 
Hispanic 


Total 


Local 
education 
agency (LEA) 


2% 


11 % 


18 % 


43 % 


26% 


100 % 


Community 
college 


1 


7 


18 


27 


47 


100 


Other 


2 % 


6% 


9 % 


18% 


65% 


100 % 
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Exhibit 43 

Mean Pretest Achievement Scores of New Clients 
by Program Sponsorship 



Indicator 


Local Public 
Education 
Agency 


Community 
College 


Other Type of 
Sponsor 


Average for 

Adult 
Education 
Population 


CASAS (ESL) 
scale score 
{N = 2^5) 


207 


211 


201 


207 


CASAS (ABE) 
scale score 
(N =: 983) 


226 


233 


229 


229 


CASAS (ASH) 
scale score 


234 


235 


231 


234 


TABE (ABE) 
grade-equivalent score 
(N = 1,484) 


7.7 


7.9 


7.0 


7.6 


TABE (ASE) 
grade-equivalent score 
(N = 2,250) 


10.0 


10.2 


11.3 


10.3 



Exhibit 4.4 
Distribution of New Clients 
by Age Group and Program Component 
(N = 18,357) 



Age 
Group 


ESL 


ABE 


ASE 


Average 


Youth group 
(ages 16-21) 


22% 


28% 


44 % 


30% 


Young adult group 
(ages 22-30) 


39 


30 


27 


33 


Middle age group 
(ages 31-45) 


29 


32 


23 


28 


Older client group 
(over age 45) 


10 


10 


6 


9 


Total 


100% 


100% 


100 % 


100 % 
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Exhibit 4.5 

Distribution of New Clients by Personal and Family Characteristics 

and Age Group 
(N = 14,109-22,548) 



Oiaracteristic 


Youth 
Group 
(ages 16-21) 


Young Adult 

Group 
(ages 22-30) 


Middle Age 

Group 
(ages 31-45) 


Older Client 

Group 
(over age 45) 


Adult 
Education 
Population 

Mean 


National origin 


Native-bom 
60% 


Foi'ign-bom 
60 % 


Foreign-bom 
57% 


Foreign-bom 
60% 


Foreign-bom 
53 % 


Sex 


Female 

50% 


Female 
56 % 


Female 
61 % 


Female 
62 % 


Female 
56% 


Language 
other than 
English spoken 
at home 


English 
52 % 


Other 
64 % 


Other 
60 % 


Other 
64% 


Other 
58% 


Young children 


None 
63 % 


None 
55 % 


None 
65% 


None 
83% 


None 
62% 



Exhibit 4.6 

Distribution of New Clients by Age Group 
and Race/Ethnicity 
(N = 16,747) 



Age Group 


American 
Indian/ Alaskan 
Native 


Asian/ 
Pacific 
Islander 


Black, 
non- 
Hispanic 


Hispanic 


White, 
non- 
Hispanic 


Total 


Youth group 
(ages 16-21) 


2 % 


5 % 


17% 


36 % 


40 % 


100 % 


Young adult 
group 

(ages 22-30) 


2 


11 


16 


43 


28 


100 


Middle age group 
(ages 31-45) 


2 


11 


20 


36 


31 


100 


Older client group 
(over age 45) 


2 % 


16% 


18% 


33 % 


31 % 


100 % 



Exhibit 4.7 

Distribution of New Clients by Employment Status 
and Age Group 
(N = 14,076) 



Indicator 


Youth Group 
(ages 16-21) 


Young Adult 

Group 
(ages 22-30) 


Middle Age 

Group 
(ages 31-45) 


Older Client 

Group 
(over age 45) 


Average for 

Adult 
Education 
Population 


Employed 


38% 


46% 


43% 


40% 


42% 


Unemployed 


29 


19 


21 


15 


22 


Not in work 
force 


33 


35 


37 


45 


35 


Total 


100 % 


100 % 


100% 


100% 


100 % 
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Exhibit 4.8 

Distribution of New Clients by Education Characteristics 

and Age Groups 



Indicator 


Youth 
Group 
(ages 16-21) 


Yoimg 
Adult Group 
(ages 22-30) 


Middle Age 

Group 
(ages 31-45 ) 


Older Client 

Group 
(Over age 45) 


for 
Adult 
Education 
Population 


Median years of 
schooling 
(N = 20,940) 


10 


10 


10 


9 


10 


Percentage without 
a high school 
diploma 
(N = 19,016) 


80 


62 


63 


68 


68 


CASAS (ESL) 
mean scale score 
(N = 2,345) 


205 


209 


208 


199 


207 


CASAS (ABE) 
mean scale score 
(N = 983) 


231 


230 


229 


217 


229 


CASAS (ASE) 
mean scale score 
(N = 1,589) 


235 


234 


235 


232 


• 234 


TABE (ABE) 
mean scale 
equivalent 
(N = 1,484 


7.5 


7.8 


7.7 


7.0 


7.6 


TABE (ASE) mean 
grade equivalent 
(N = 2,250) 


10.3 


10.4 


10.4 


9.9 


10.3 
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Exhibit 4.9 
Distribution of New Clients 
by Type of Community and Program Component 
(N = 20,418) 



Type of Community 


ESL 


ABE 


ASE 


Average 


Large city in major metropolitan area 


54 % 


13 % 


6 % 


30 % 


Remainder of major metropolitan area 


31 


30 


26 


30 


Small metropolitan area 


11 


27 


33 


21 


Nonmetropolitan area 


4 


30 


35 


19 


Total 


100 % 


100% 


100 % 


100 % 



Note: major metropolitan area was defined as having a population of 1.5 million or 
more; large cities in major metropolitan areas as having a population of 500,000 or more; 
and small metropolitan areas as any community located within a standard metropolitan 
area with a population of less than 1.5 inillion. 



Exhibit 4.10 

Distribution of New Clients by Race/Ethnicity and Type of Community 

{N = 22,067) 



Race/ 
Ethnicity 


Large City 
in Major 
Metro 
Area 


Remainder 
of 

Major Metro 
Area 


Small 
Metro 
Area 


Nonmetro 
Area 


1 

Average 

for 
Ethnic' ty 


American 
Indian 


<1 % 


1 % 


2 % 


6% 


2 % 


Asian/Pacific 
Islander 


13 


10 


9 


4 


10 


Black, non- 
Hispanic 


9 


31 


14 


14 


17 


Hispanic 


70 


43 


16 


8 


38 


White, non- 
Hispanic 


8 


16 


58 


68 


33 


Average for 

Type of 
Community 


30 % 

1 


29 % 


21 % 


19 % 


100 % 
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Exhibit 4.11 

Distribution of New Clients by Census Region and Type of Community 

(N = 22,548) 



Census Region 


Large City in 
Major Metro 
Area 


Remainder of 

IVldJUr iVicliU 

Area 


Small Metro 

A T*PP 


Nonmetro 
Area 


Total Region 


Northeast 


15% 


26 % 


55 % 


4 % 


11 % 


North Central 


27 


10 


27 


36 


19 


South 


1 


47 


21 


32 


30 


West 


57 


26 


9 


8 


40 


U.S. Ave'-age 


29 % 


30 % 


21 % 


20 % 


100% 



Exhibit 4.12 

Mean Pretest Achievement Scores of New Clients by 
Type of Community 



Indicator 


Large City in 
Major Metro 
Area 


Remainder of 
Major Metro 
Area 


Small 
Metro Area 


Nonmetro 
Area 


Average 
for Adult 
Education 
Population 


CASAS (ESL) 

scale score (N=2,345) 


206 


208 


208 


211 


207 


CASAS (ABE) 
scale score (N=983) 


232 


223 


230 


229 


229 


CASAS (ASE) 

scale score (N=l,589) 


235 


236 


235 


234 


234 


TABE (ABE) 
grade-equivalent score 
(N=l,484) 


7.0 


6.2 


7.2 


8.4 


7.6 


TABE (ASE) 
grade-equivalent score 
(N=2,250) 


9.2 


10.0 


10.5 


10.4 


10.3 
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Exhibit 4.13 

Expected and Actual Placement for Clients Based on Test Scores 
(N = 2,345 for CASAS ESL; 2,572 for CASAS ABE/ASE; 3,734 for TABE) 



Placement 


Percentage of CASAS 
Sub-Sample 


Percentage of TABE 
Sub-Sample 


Level 


Expected 


Actual 


Expected 


Actual 


ESL Beginning 


33 % 


78 % 






ESL Intermediate 


47 


14 






ESL Advanced 


20 


8 






ABE Beginning 


10 % 


15 % 


14% 


17% 


ABE Intermediate 


12 


23 


32 


23 


ASE 


78 


62 


54 


60 



Exhibit 4.14 
Appropriateness of New Clients Placement 
Mean Pre-Test scores by Program Placement Level* 
(Ns range from 284-3,740, depending on the cell) 



Initial 
Program 
Placemeiit 


Expected Scores 


Percent 
Appropriately 
Placed 


Mean 
CASAS Scale 
Score 


Mean TABE 

Grade- 
Equivalent 
Score 


CASAS 


TABE 


ESL 

Beginning 


181-200 




34 % 


203.0 




ESL 

Intermediate 


201-215 




34 


217.5 




ESL 

Advanced 


216-224 




20 


226.7 




ABE 

Beginning 


214 or 
below 


6.0 or 
below 


38 


225.1 


6.6 


ABE 

Intermediate 


215-224 


6.1-8.9 


42 


231.6 


8.3 


ASE/GED 


225 or 
above 


9.0 or 
above 


76 % 


234.4 


10.3 



^Appropriate placement is defined as having a pretest score falling within the "Expected 
Scores" range for the level at which clients began receiving services. 
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Exhibit 4.15 

Comparison of Past Educational Experience of the Adult Education 
Program Target Population and New Client Population 



Years of School Completed 


Target Population 


New Clients 


o or less 


4U /o 


29 % 


9 


13 


18 


10 


17 ■ 


17 


11 


15 


23 


12 


15 


13 


Total 


100 % 


100 % 



Note: The "target population" is defined as individuals aged 16 years and older who 
have not attained a high school diploma or equivalent and are not currently enrolled in 
school To be more equivalent to the target population data, the adult education client 
population represented in this table excludes ABE/ASE clients (about 6 percent of all 
clients) with a high school diploma or equivalent. The Adult Education Act permits 
serving clients with a high sdiool diploma if they meet other criteria of need. 
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Exhibit 4.16 

Comparison of Past Educational Experience of Adult Education Program Target 
Population and of the Program's New Client Population 



Years of 
School 
oitidI eted 


16-24 years 


25-44 years 


45-59 years 


60 years and 
older 


Target 
Pop. 


New 
Clients 


Target 
Pop. 


New 
Clients 


Target 
Pop. 


New 
Clients 


Target 
Pop. 


New 
Clients 


8 or less 


20 % 


21 % 


28 % 


33% 


38 % 


48% 


53 % 


59% 


9 


15 


20 


13 


17 


13 


13 


12 


10 


10 


22 


19 


19 


16 


18 


12 


14 


7 


11 


23 


28 


19 


21 


16 


14 


10 


16 


12 (no 
diploma) 


20 


13 


21 


13 


15 


12 


11 


8 


Total 


100 % 


100% 


100 % 


100 % 


100% 


100% 


100 % 


100 % 



Note- The "target population" is defined as individuals aged 16 years and older who have not 
attained a high school diploma or equivalent and are not currently enroUed in school. To be more 
eqmvalent to the target population data, the adult education cUent population represented in this table 
excludes ABE/ASE cHents (about 6 percent of all cUents) witli a high school diploma or equivalent. 
The Adult Education Act permits serving cHents with a high school diploma if they meet other critena 
of need. 



Exhibit 4.17 

Comparison of English Speaking Ability of ESL Clients and 

Target Population 



English Speaking 
Ability 


Total ESL 
Population 


New ESL 
Clients 


Ratio of Percent New 
Clients to Percent Target 
Populaton 


Not at All 


14% 


25 % 


1.8 


Not Well 


25 


62 


2.5 


WeU 


25 


n 


.4 


Very WeU 


36 


2 


.1 


Total 


100 % 


100 % 
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